[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:15 dist-test-slave-6zbq krb5kdc[30415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:15 dist-test-slave-6zbq krb5kdc[30415](info): set up 2 sockets
Aug 10 18:48:15 dist-test-slave-6zbq krb5kdc[30415](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 18:48:16.688356 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:16 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:16.878293 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33123
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:33123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:17.556468 30437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:17.556900 30437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:17.557127 30437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:17.566819 30437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:17.567021 30437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:17.567162 30437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:17.567307 30437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:17.567421 30437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:17.580158 30437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:33123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:17.582170 30437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:17.584367 30437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:17.604686 30443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.604815 30444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:17.605430 30449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:17.606928 30437 server_base.cc:1034] running on GCE node
I20260810 18:48:17.608099 30437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:17.610423 30437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:17.611685 30437 hybrid_clock.cc:648] HybridClock initialized: now 1786387697611586 us; error 125 us; skew 500 ppm
I20260810 18:48:17.612318 30437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:17 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387697, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:17.620878 30437 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:17.623942 30437 webserver.cc:460] Webserver started at http://127.29.174.254:32855/ using document root <none> and password file <none>
I20260810 18:48:17.625201 30437 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:17.625326 30437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:17.625813 30437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:17.630455 30437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "6e4bfcda9c3d4babb6c879b855fc55bc"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-6zbq"
server_key: "ffc03467db556224d1275596b1fa4a32"
server_key_iv: "a7bcac97243025234398b585eb1ff956"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.631409 30437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "6e4bfcda9c3d4babb6c879b855fc55bc"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-6zbq"
server_key: "ffc03467db556224d1275596b1fa4a32"
server_key_iv: "a7bcac97243025234398b585eb1ff956"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.640177 30437 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.002s
I20260810 18:48:17.646703 30459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:17.648626 30437 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 18:48:17.648891 30437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6e4bfcda9c3d4babb6c879b855fc55bc"
format_stamp: "Formatted at 2026-08-10 18:48:17 on dist-test-slave-6zbq"
server_key: "ffc03467db556224d1275596b1fa4a32"
server_key_iv: "a7bcac97243025234398b585eb1ff956"
server_key_version: "encryptionkey@0"
I20260810 18:48:17.649175 30437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:17.679066 30437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:17.685349 30437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:17.685734 30437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:17.726696 30437 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:33123
I20260810 18:48:17.726754 30558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:33123 every 8 connection(s)
I20260810 18:48:17.729277 30437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:17.733896 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30437
I20260810 18:48:17.734252 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:17.736629 30395 external_mini_cluster.cc:1428] Setting key d5ea1e4df17f480efb0d7fbc9bd06018
I20260810 18:48:17.737188 30559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:17.756172 30559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc: Bootstrap starting.
I20260810 18:48:17.764281 30559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:17.766469 30559 log.cc:824] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:17.771499 30559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc: No bootstrap required, opened a new log
Aug 10 18:48:17 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:17.785283 30559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } }
I20260810 18:48:17.785884 30559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:17.786062 30559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4bfcda9c3d4babb6c879b855fc55bc, State: Initialized, Role: FOLLOWER
I20260810 18:48:17.787493 30559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } }
I20260810 18:48:17.787992 30559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:17.788283 30559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:17.788563 30559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:17.790982 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:17.742901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.29.174.254:33123)
0810 18:48:17.744578 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:17.744595 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:17.748347 (+ 3752us) server_negotiation.cc:407] Connection header received
0810 18:48:17.762034 (+ 13687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:17.762061 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:17.762837 (+ 776us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:17.763343 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:17.766942 (+ 3599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.768822 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.770938 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:17.772222 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:17.772479 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:17.779378 (+ 6899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:17.779451 (+ 73us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:17.779497 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:17.779573 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:17.784624 (+ 5051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.785417 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.785426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.785435 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.785740 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:17.786212 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:17.786217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:17.786221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:17.787244 (+ 1023us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:17.787695 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:17.788573 (+ 878us) server_negotiation.cc:299] Negotiation successful
0810 18:48:17.789027 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":833,"threads_started":1}
I20260810 18:48:17.792165 30559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } }
I20260810 18:48:17.793193 30559 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4bfcda9c3d4babb6c879b855fc55bc; no voters:
I20260810 18:48:17.794291 30559 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:17.794704 30567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:17.797610 30567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [term 1 LEADER]: Becoming Leader. State: Replica: 6e4bfcda9c3d4babb6c879b855fc55bc, State: Running, Role: LEADER
I20260810 18:48:17.798717 30559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:17.798599 30567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } }
I20260810 18:48:17.805944 30573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4bfcda9c3d4babb6c879b855fc55bc. Latest consensus state: current_term: 1 leader_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } } }
I20260810 18:48:17.806649 30573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:17.805735 30568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4bfcda9c3d4babb6c879b855fc55bc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33123 } } }
I20260810 18:48:17.807252 30568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:17.812222 30589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:17.822139 30589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:17.837800 30589 catalog_manager.cc:1347] Generated new cluster ID: 7c0c7df23ef9497699ac1db76a02ac91
I20260810 18:48:17.837950 30589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:17.875391 30589 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:17.877070 30589 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:17.888195 30589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e4bfcda9c3d4babb6c879b855fc55bc: Generated new TSK 0
I20260810 18:48:17.889125 30589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:17.948231 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:18.507081 30607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:18.507465 30607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:18.507699 30607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:18.517516 30607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:18.517632 30607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:18.517817 30607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:18.530526 30607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:18.531836 30607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:18.533217 30607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:18.549678 30616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:18.549880 30618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:18.549888 30624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:18.551793 30607 server_base.cc:1034] running on GCE node
I20260810 18:48:18.552434 30607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:18.553308 30607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:18.554493 30607 hybrid_clock.cc:648] HybridClock initialized: now 1786387698554438 us; error 48 us; skew 500 ppm
I20260810 18:48:18.554819 30607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:18 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:18.557917 30607 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:18.559856 30607 webserver.cc:460] Webserver started at http://127.29.174.193:40905/ using document root <none> and password file <none>
I20260810 18:48:18.560485 30607 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:18.560597 30607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:18.560887 30607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:18.564091 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "7623fca46ab040d19ac074e18a2fa60c"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-6zbq"
server_key: "fe053f06dc2870645f179dfc2bf7e19e"
server_key_iv: "426f80289815667bef29f6db8b8c3ae9"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.564858 30607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "7623fca46ab040d19ac074e18a2fa60c"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-6zbq"
server_key: "fe053f06dc2870645f179dfc2bf7e19e"
server_key_iv: "426f80289815667bef29f6db8b8c3ae9"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.571260 30607 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260810 18:48:18.575874 30642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:18.577267 30607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:48:18.577469 30607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "7623fca46ab040d19ac074e18a2fa60c"
format_stamp: "Formatted at 2026-08-10 18:48:18 on dist-test-slave-6zbq"
server_key: "fe053f06dc2870645f179dfc2bf7e19e"
server_key_iv: "426f80289815667bef29f6db8b8c3ae9"
server_key_version: "encryptionkey@0"
I20260810 18:48:18.577698 30607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:18.597997 30607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:18.603542 30607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:18.603796 30607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:18.605532 30607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:18.608417 30607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:18.608513 30607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:18.608696 30607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:18.608784 30607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:18.682521 30607 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:42097
I20260810 18:48:18.682833 30832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:42097 every 8 connection(s)
I20260810 18:48:18.684139 30607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:18.691285 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30607
I20260810 18:48:18.691627 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:18.692662 30395 external_mini_cluster.cc:1428] Setting key d42f152cf6025a4e753db7d601ddcbb4
Aug 10 18:48:18 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387698, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:18.713219 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.688100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:45463 (local address 127.29.174.254:33123)
0810 18:48:18.688631 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:18.688635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:18.689874 (+ 1239us) server_negotiation.cc:407] Connection header received
0810 18:48:18.693247 (+ 3373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:18.693252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:18.693382 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:18.693497 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:18.695723 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.696697 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.699612 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.700186 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.700280 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.705675 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:18.705680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:18.705684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:18.705715 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:18.709498 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.710371 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.710377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.710381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.710609 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:18.711145 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:18.711149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:18.711152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:18.711469 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:18.711609 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:18.712408 (+ 799us) server_negotiation.cc:299] Negotiation successful
0810 18:48:18.712668 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":285,"threads_started":1}
I20260810 18:48:18.713966 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:18.688780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33123 (local address 127.29.174.193:45463)
0810 18:48:18.689715 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:18.689761 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:18.692922 (+ 3161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:18.693761 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:18.693773 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:18.694610 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:18.695520 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.695558 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.696885 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.696891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.699465 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:18.699484 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:18.700488 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:18.700498 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:18.700949 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:18.701821 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:18.701856 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:18.705378 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:18.709630 (+ 4252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.709637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.709650 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.710214 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.710707 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:18.710711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:18.710713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:18.711009 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:18.711581 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:18.711587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:18.711948 (+ 361us) client_negotiation.cc:769] Sending connection context
0810 18:48:18.712234 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 18:48:18.712678 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":372,"threads_started":1}
I20260810 18:48:18.715744 30835 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33123
I20260810 18:48:18.716338 30835 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:18.717775 30835 heartbeater.cc:511] Master 127.29.174.254:33123 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
I20260810 18:48:18.722527 30497 ts_manager.cc:194] Registered new tserver with Master: 7623fca46ab040d19ac074e18a2fa60c (127.29.174.193:42097)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
I20260810 18:48:18.725376 30497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:45463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:18.751619 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:19.294795 30843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:19.295167 30843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:19.295387 30843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:19.305092 30843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:19.305289 30843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:19.305505 30843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:19.318441 30843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:19.320094 30843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:19.321784 30843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:19.338802 30852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:19.339519 30856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:19.339789 30853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:19.340912 30843 server_base.cc:1034] running on GCE node
I20260810 18:48:19.341660 30843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:19.342792 30843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:19.344043 30843 hybrid_clock.cc:648] HybridClock initialized: now 1786387699343997 us; error 34 us; skew 500 ppm
I20260810 18:48:19.344466 30843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:19 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387699, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:19.348433 30843 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:19.350241 30843 webserver.cc:460] Webserver started at http://127.29.174.194:37425/ using document root <none> and password file <none>
I20260810 18:48:19.351114 30843 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:19.351287 30843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:19.351609 30843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:19.355129 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "e971b8e33b4149e4970656a8b9738571"
format_stamp: "Formatted at 2026-08-10 18:48:19 on dist-test-slave-6zbq"
server_key: "2c80e0181f8e368749e0a2115a47277b"
server_key_iv: "444dc286b6b04d126c1818bef04ffbdf"
server_key_version: "encryptionkey@0"
I20260810 18:48:19.355988 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "e971b8e33b4149e4970656a8b9738571"
format_stamp: "Formatted at 2026-08-10 18:48:19 on dist-test-slave-6zbq"
server_key: "2c80e0181f8e368749e0a2115a47277b"
server_key_iv: "444dc286b6b04d126c1818bef04ffbdf"
server_key_version: "encryptionkey@0"
I20260810 18:48:19.362239 30843 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260810 18:48:19.366822 30870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:19.368266 30843 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:48:19.368469 30843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "e971b8e33b4149e4970656a8b9738571"
format_stamp: "Formatted at 2026-08-10 18:48:19 on dist-test-slave-6zbq"
server_key: "2c80e0181f8e368749e0a2115a47277b"
server_key_iv: "444dc286b6b04d126c1818bef04ffbdf"
server_key_version: "encryptionkey@0"
I20260810 18:48:19.368737 30843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:19.404309 30843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:19.409442 30843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:19.409854 30843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:19.411360 30843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:19.413352 30843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:19.413448 30843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:19.413648 30843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:19.413743 30843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:19.485577 30843 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:39489
I20260810 18:48:19.485646 31055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:39489 every 8 connection(s)
I20260810 18:48:19.487218 30843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:19.492834 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30843
I20260810 18:48:19.493171 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:19.494045 30395 external_mini_cluster.cc:1428] Setting key 06aaca3235a41cad63ca883b706d0d51
Aug 10 18:48:19 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387699, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:19.514628 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:19.490516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:50935 (local address 127.29.174.254:33123)
0810 18:48:19.491142 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:19.491149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:19.492473 (+ 1324us) server_negotiation.cc:407] Connection header received
0810 18:48:19.495852 (+ 3379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:19.495856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:19.495921 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:19.496001 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:19.498396 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:19.499441 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:19.502441 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:19.503098 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:19.503198 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:19.508002 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:19.508007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:19.508011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:19.508043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:19.511415 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:19.512184 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:19.512189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:19.512193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:19.512316 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:19.512810 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:19.512813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:19.512815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:19.513071 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:19.513208 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:19.514011 (+ 803us) server_negotiation.cc:299] Negotiation successful
0810 18:48:19.514243 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":388,"threads_started":1}
I20260810 18:48:19.515964 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:19.491320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33123 (local address 127.29.174.194:50935)
0810 18:48:19.492303 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:19.492340 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:19.495512 (+ 3172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:19.496246 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:19.496253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:19.497186 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:19.498192 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:19.498220 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:19.499638 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:19.499643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:19.502282 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:19.502297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:19.503394 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:19.503404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:19.503724 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:19.504539 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:19.504602 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:19.507791 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:19.511539 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:19.511548 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:19.511563 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:19.512052 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:19.512416 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:19.512421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:19.512425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:19.512695 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:19.513191 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:19.513198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:19.513551 (+ 353us) client_negotiation.cc:769] Sending connection context
0810 18:48:19.513967 (+ 416us) client_negotiation.cc:240] Negotiation successful
0810 18:48:19.514564 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":464,"threads_started":1}
I20260810 18:48:19.517400 31057 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33123
I20260810 18:48:19.517706 31057 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
I20260810 18:48:19.518772 31057 heartbeater.cc:511] Master 127.29.174.254:33123 requested a full tablet report, sending...
I20260810 18:48:19.520455 30497 ts_manager.cc:194] Registered new tserver with Master: e971b8e33b4149e4970656a8b9738571 (127.29.174.194:39489)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
I20260810 18:48:19.521668 30497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:50935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:19.553224 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:19.730285 30835 heartbeater.cc:503] Master 127.29.174.254:33123 was elected leader, sending a full tablet report...
W20260810 18:48:20.108723 31070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:20.108987 31070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:20.109094 31070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:20.117627 31070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:20.117729 31070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:20.117847 31070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:20.129089 31070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33123
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:20.130353 31070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:20.131948 31070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:20.147948 31082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:20.147970 31089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:20.148054 31085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:20.149474 31070 server_base.cc:1034] running on GCE node
I20260810 18:48:20.149979 31070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:20.150856 31070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:20.152045 31070 hybrid_clock.cc:648] HybridClock initialized: now 1786387700152010 us; error 27 us; skew 500 ppm
I20260810 18:48:20.152359 31070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:20.156317 31070 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:20.157876 31070 webserver.cc:460] Webserver started at http://127.29.174.195:45919/ using document root <none> and password file <none>
I20260810 18:48:20.158754 31070 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:20.158902 31070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:20.159235 31070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:20.162672 31070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "cd518d0e9dd94d5db3ecca83ebe02c3a"
format_stamp: "Formatted at 2026-08-10 18:48:20 on dist-test-slave-6zbq"
server_key: "3de0fc52a0e520f3b65885c8b36c49d8"
server_key_iv: "d87f3667b3d6bcafc1db9627ee74d1d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:20.163439 31070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "cd518d0e9dd94d5db3ecca83ebe02c3a"
format_stamp: "Formatted at 2026-08-10 18:48:20 on dist-test-slave-6zbq"
server_key: "3de0fc52a0e520f3b65885c8b36c49d8"
server_key_iv: "d87f3667b3d6bcafc1db9627ee74d1d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:20.169740 31070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 18:48:20.173919 31105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:20.175299 31070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:20.175486 31070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "cd518d0e9dd94d5db3ecca83ebe02c3a"
format_stamp: "Formatted at 2026-08-10 18:48:20 on dist-test-slave-6zbq"
server_key: "3de0fc52a0e520f3b65885c8b36c49d8"
server_key_iv: "d87f3667b3d6bcafc1db9627ee74d1d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:20.175673 31070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:20.210141 31070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:20.215286 31070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:20.215636 31070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:20.217624 31070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:20.220239 31070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:20.220359 31070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:20.220960 31070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:20.221086 31070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:20.291746 31070 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:36199
I20260810 18:48:20.291949 31315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:36199 every 8 connection(s)
I20260810 18:48:20.293179 31070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:20.300542 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31070
I20260810 18:48:20.300836 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:20.301517 30395 external_mini_cluster.cc:1428] Setting key 17cad6788acf0ad99c72afe2994663f2
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:20.319913 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.296448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:38303 (local address 127.29.174.254:33123)
0810 18:48:20.297288 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.297296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.298199 (+ 903us) server_negotiation.cc:407] Connection header received
0810 18:48:20.301010 (+ 2811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.301014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.301104 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.301338 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.303982 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.305029 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.307534 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.307986 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.308079 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.312241 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.312246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.312249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.312289 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.316060 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.316866 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.316869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.316871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.317046 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.317654 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.317658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.317660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.318104 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.318291 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.319239 (+ 948us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.319451 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":525,"threads_started":1}
I20260810 18:48:20.320844 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.297122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33123 (local address 127.29.174.195:38303)
0810 18:48:20.298000 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:20.298043 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:20.300722 (+ 2679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:20.301663 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:20.301674 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:20.302847 (+ 1173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:20.303778 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:20.303805 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.305236 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.305242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:20.307413 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:20.307426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.308261 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.308270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:20.308577 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.309217 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:20.309240 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:20.311946 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:20.316210 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:20.316217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:20.316231 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:20.316766 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:20.317152 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:20.317156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:20.317172 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:20.317518 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:20.318235 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:20.318243 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:20.318726 (+ 483us) client_negotiation.cc:769] Sending connection context
0810 18:48:20.319065 (+ 339us) client_negotiation.cc:240] Negotiation successful
0810 18:48:20.319544 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":453,"threads_started":1}
I20260810 18:48:20.322674 31316 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33123
I20260810 18:48:20.322993 31316 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:20.323686 31316 heartbeater.cc:511] Master 127.29.174.254:33123 requested a full tablet report, sending...
I20260810 18:48:20.325258 30497 ts_manager.cc:194] Registered new tserver with Master: cd518d0e9dd94d5db3ecca83ebe02c3a (127.29.174.195:36199)
I20260810 18:48:20.326526 30497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:38303
I20260810 18:48:20.331585 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:20.393456 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.370360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.29.174.193:42097)
0810 18:48:20.371566 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.371576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.372707 (+ 1131us) server_negotiation.cc:407] Connection header received
0810 18:48:20.372928 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.372937 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.373246 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.373466 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.376094 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.378598 (+ 2504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.380493 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.381605 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.381747 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.386997 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.387009 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.387024 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.387058 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.390784 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.391416 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.391423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.391425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.391629 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.391913 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.391916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.391918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.392344 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.392667 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.392806 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.393027 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":682,"threads_started":1}
W20260810 18:48:20.394855 30689 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39548
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:20.449335 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.434792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.29.174.193:42097)
0810 18:48:20.434958 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.434962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.435411 (+ 449us) server_negotiation.cc:407] Connection header received
0810 18:48:20.435565 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.435568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.435669 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.435755 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.436977 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.437746 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.439240 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.439643 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.439743 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.444985 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.444990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.444993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.445025 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.447468 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.447985 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.447988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.447990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.448109 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.448368 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.448371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.448373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.448588 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.448754 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.448940 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.449093 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:48:20.451253 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30607
I20260810 18:48:20.465201 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.452367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.29.174.193:42097)
0810 18:48:20.452551 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.452556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.452694 (+ 138us) server_negotiation.cc:407] Connection header received
0810 18:48:20.452802 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.452805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.452857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.452927 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.454572 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.455247 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.456890 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.457494 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.457581 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.459915 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.459920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.459923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.459949 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.463002 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.463642 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.463645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.463647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.463799 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.464143 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.464146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.464148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.464407 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.464565 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.464793 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.464928 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:20.465926 30826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:20.524181 31057 heartbeater.cc:503] Master 127.29.174.254:33123 was elected leader, sending a full tablet report...
I20260810 18:48:20.553778 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30607
I20260810 18:48:20.574501 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30843
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:20.589869 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.575318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.29.174.194:39489)
0810 18:48:20.576117 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.576124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.576149 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:48:20.576225 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.576231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.576448 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.576569 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.577913 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.578990 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.580145 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.580759 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.580854 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.584743 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.584754 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.584767 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.584804 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.587562 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.588109 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.588115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.588117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.588250 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.588539 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.588541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.588543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.588854 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.589075 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.589301 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.589490 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":474,"threads_started":1}
I20260810 18:48:20.591147 31050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:20.675256 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30843
I20260810 18:48:20.696732 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31070
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[30415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:20.712327 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:20.697647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.29.174.195:36199)
0810 18:48:20.698156 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:20.698163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:20.698194 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:48:20.698259 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:20.698266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:20.698513 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:20.698661 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:20.700285 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.701260 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.702775 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:20.703314 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:20.703429 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:20.707230 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:20.707241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:20.707253 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:20.707285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:20.709755 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.710364 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.710373 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.710376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.710497 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:20.710839 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:20.710844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:20.710848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:20.711169 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:20.711448 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:20.711754 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 18:48:20.712009 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":218,"threads_started":1}
I20260810 18:48:20.713527 31308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:20.793542 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31070
I20260810 18:48:20.810225 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30437
I20260810 18:48:20.811363 30551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:20.909427 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30437
2026-08-10T18:48:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5552 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[31367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[31367](info): set up 2 sockets
Aug 10 18:48:20 dist-test-slave-6zbq krb5kdc[31367](info): commencing operation
krb5kdc: starting...
W20260810 18:48:22.107432 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:22 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:22.283517 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43701
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:43701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:22.606660 31392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:22.607046 31392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:22.607183 31392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:22.615825 31392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:22.615921 31392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:22.615991 31392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:22.616072 31392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:22.616132 31392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:22.625552 31392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:43701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:22.626840 31392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:22.628057 31392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:22.639046 31399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:22.639003 31401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:22.639104 31398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:22.639873 31392 server_base.cc:1034] running on GCE node
I20260810 18:48:22.640640 31392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:22.641921 31392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:22.643110 31392 hybrid_clock.cc:648] HybridClock initialized: now 1786387702643064 us; error 36 us; skew 500 ppm
I20260810 18:48:22.643421 31392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:22 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387702, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:22.649374 31392 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:22.650734 31392 webserver.cc:460] Webserver started at http://127.29.174.254:35099/ using document root <none> and password file <none>
I20260810 18:48:22.651353 31392 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:22.651466 31392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:22.651820 31392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:22.654675 31392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "d50288900c214c60a7fbd794d66df700"
format_stamp: "Formatted at 2026-08-10 18:48:22 on dist-test-slave-6zbq"
server_key: "4ee9b4f467a25238f69c8f4c77f27d26"
server_key_iv: "d448d7a53da6e3d37a23bf91c28d8e5a"
server_key_version: "encryptionkey@0"
I20260810 18:48:22.655390 31392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "d50288900c214c60a7fbd794d66df700"
format_stamp: "Formatted at 2026-08-10 18:48:22 on dist-test-slave-6zbq"
server_key: "4ee9b4f467a25238f69c8f4c77f27d26"
server_key_iv: "d448d7a53da6e3d37a23bf91c28d8e5a"
server_key_version: "encryptionkey@0"
I20260810 18:48:22.661141 31392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:48:22.665539 31411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:22.667770 31392 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260810 18:48:22.668012 31392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "d50288900c214c60a7fbd794d66df700"
format_stamp: "Formatted at 2026-08-10 18:48:22 on dist-test-slave-6zbq"
server_key: "4ee9b4f467a25238f69c8f4c77f27d26"
server_key_iv: "d448d7a53da6e3d37a23bf91c28d8e5a"
server_key_version: "encryptionkey@0"
I20260810 18:48:22.668267 31392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:22.706588 31392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:22.710414 31392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:22.710659 31392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:22.727404 31392 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:43701
I20260810 18:48:22.727434 31491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:43701 every 8 connection(s)
I20260810 18:48:22.728663 31392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:22.732182 31493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:22.738190 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31392
I20260810 18:48:22.738456 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:22.739352 30395 external_mini_cluster.cc:1428] Setting key 64c39ede4d887812dcb6a5665dd8570c
I20260810 18:48:22.743011 31493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700: Bootstrap starting.
I20260810 18:48:22.746565 31493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:22.747558 31493 log.cc:824] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:22.750283 31493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700: No bootstrap required, opened a new log
Aug 10 18:48:22 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:22.758154 31493 raft_consensus.cc:348] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } }
I20260810 18:48:22.758548 31493 raft_consensus.cc:374] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:22.758658 31493 raft_consensus.cc:729] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50288900c214c60a7fbd794d66df700, State: Initialized, Role: FOLLOWER
I20260810 18:48:22.759186 31493 consensus_queue.cc:260] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } }
I20260810 18:48:22.759415 31493 raft_consensus.cc:388] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:22.759547 31493 raft_consensus.cc:482] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:22.759725 31493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:22.761750 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:22.741375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.29.174.254:43701)
0810 18:48:22.742074 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:22.742086 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:22.742125 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:48:22.743847 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:22.743869 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:22.744206 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:22.744532 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:22.746535 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:22.747841 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:22.749340 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:22.749958 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:22.750082 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:22.754442 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:22.754450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:22.754467 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:22.754512 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:22.757367 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:22.757995 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:22.758001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:22.758007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:22.758161 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:22.758540 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:22.758544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:22.758546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:22.759151 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:22.759441 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:22.759967 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 18:48:22.760407 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":323,"threads_started":1}
I20260810 18:48:22.762784 31493 raft_consensus.cc:504] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } }
I20260810 18:48:22.763300 31493 leader_election.cc:302] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d50288900c214c60a7fbd794d66df700; no voters:
I20260810 18:48:22.763933 31493 leader_election.cc:288] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:22.764001 31502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:22.765673 31493 sys_catalog.cc:564] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:22.767154 31502 raft_consensus.cc:686] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [term 1 LEADER]: Becoming Leader. State: Replica: d50288900c214c60a7fbd794d66df700, State: Running, Role: LEADER
I20260810 18:48:22.767693 31502 consensus_queue.cc:237] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } }
I20260810 18:48:22.770385 31503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d50288900c214c60a7fbd794d66df700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } } }
I20260810 18:48:22.770838 31503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:22.770805 31505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d50288900c214c60a7fbd794d66df700. Latest consensus state: current_term: 1 leader_uuid: "d50288900c214c60a7fbd794d66df700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d50288900c214c60a7fbd794d66df700" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43701 } } }
I20260810 18:48:22.771071 31505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:22.772429 31520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:22.776304 31520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:22.783499 31520 catalog_manager.cc:1347] Generated new cluster ID: 5f3fdcfeb6ec4522b9e0ab717f2c478a
I20260810 18:48:22.783595 31520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:22.791971 31520 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:22.793623 31520 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:22.801438 31520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d50288900c214c60a7fbd794d66df700: Generated new TSK 0
I20260810 18:48:22.802248 31520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:22.851250 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:23.173776 31531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:23.174039 31531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:23.174172 31531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:23.181135 31531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:23.181228 31531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:23.181381 31531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:23.190474 31531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:23.191555 31531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:23.192761 31531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:23.204591 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.204646 31540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.204677 31548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:23.205322 31531 server_base.cc:1034] running on GCE node
I20260810 18:48:23.205837 31531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:23.206722 31531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:23.207913 31531 hybrid_clock.cc:648] HybridClock initialized: now 1786387703207863 us; error 43 us; skew 500 ppm
I20260810 18:48:23.208177 31531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:23 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:23.211061 31531 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:23.212368 31531 webserver.cc:460] Webserver started at http://127.29.174.193:36395/ using document root <none> and password file <none>
I20260810 18:48:23.212951 31531 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:23.213047 31531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:23.213289 31531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:23.215971 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "42f91657d18a4294bbe81a3a8f3f9baf"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "6c30037e4ddf0f9cb4cd65d5b9548a9b"
server_key_iv: "6ff7b2383edfd4a0892e386dcd4448d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.216734 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "42f91657d18a4294bbe81a3a8f3f9baf"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "6c30037e4ddf0f9cb4cd65d5b9548a9b"
server_key_iv: "6ff7b2383edfd4a0892e386dcd4448d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.221400 31531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:23.224557 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.225730 31531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:23.225883 31531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "42f91657d18a4294bbe81a3a8f3f9baf"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "6c30037e4ddf0f9cb4cd65d5b9548a9b"
server_key_iv: "6ff7b2383edfd4a0892e386dcd4448d4"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.226048 31531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:23.248281 31531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:23.252081 31531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:23.252332 31531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:23.253245 31531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:23.254513 31531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:23.254595 31531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.254711 31531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:23.254763 31531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.284610 31531 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:33523
I20260810 18:48:23.284672 31759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:33523 every 8 connection(s)
I20260810 18:48:23.285802 31531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:23.286954 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31531
I20260810 18:48:23.287178 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:23.287969 30395 external_mini_cluster.cc:1428] Setting key 461a295467f525b69ee74fff937ea0b1
Aug 10 18:48:23 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:23.308645 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:23.288471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:38317 (local address 127.29.174.254:43701)
0810 18:48:23.289145 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:23.289151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:23.290409 (+ 1258us) server_negotiation.cc:407] Connection header received
0810 18:48:23.291943 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:23.291949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:23.292026 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:23.292120 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:23.293990 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.294805 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.297353 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.297771 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.297897 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:23.302531 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:23.302542 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:23.302547 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:23.302591 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:23.305505 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.306292 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.306298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.306302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.306460 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.306890 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.306895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.306899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.307183 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:23.307321 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:23.308042 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 18:48:23.308294 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":407,"threads_started":1}
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
I20260810 18:48:23.309669 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:23.289077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43701 (local address 127.29.174.193:38317)
0810 18:48:23.289919 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:23.289969 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:23.291680 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:23.292361 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:23.292369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:23.292889 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:23.293766 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:23.293803 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.294971 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.294976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:23.297146 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:23.297164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.298035 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.298042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:23.298299 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:23.299184 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:23.299211 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:23.302114 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:23.305670 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:23.305679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:23.305693 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:23.306159 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:23.306568 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:23.306574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:23.306576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:23.306779 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:23.307321 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:23.307330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:23.307675 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 18:48:23.307936 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 18:48:23.308496 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":295,"threads_started":1}
I20260810 18:48:23.311200 31760 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43701
I20260810 18:48:23.311481 31760 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
I20260810 18:48:23.312223 31760 heartbeater.cc:511] Master 127.29.174.254:43701 requested a full tablet report, sending...
I20260810 18:48:23.314183 31435 ts_manager.cc:194] Registered new tserver with Master: 42f91657d18a4294bbe81a3a8f3f9baf (127.29.174.193:33523)
I20260810 18:48:23.315586 31435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:38317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:23.333303 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:23.651800 31772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:23.652041 31772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:23.652174 31772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:23.658965 31772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:23.659054 31772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:23.659210 31772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:23.668417 31772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:23.669510 31772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:23.670734 31772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:23.682718 31779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.682634 31780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:23.682674 31784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:23.683604 31772 server_base.cc:1034] running on GCE node
I20260810 18:48:23.684121 31772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:23.684944 31772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:23.686177 31772 hybrid_clock.cc:648] HybridClock initialized: now 1786387703686111 us; error 41 us; skew 500 ppm
I20260810 18:48:23.686488 31772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:23 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:23.689263 31772 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:23.690732 31772 webserver.cc:460] Webserver started at http://127.29.174.194:42957/ using document root <none> and password file <none>
I20260810 18:48:23.691339 31772 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:23.691452 31772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:23.691726 31772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:23.694326 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "8280d3a7ab31470c95fcb5f46f332595"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "9e6577a9116240960f39c4ce5e0af6ad"
server_key_iv: "fdff831f8b605d3f22625a164eaceb14"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.695227 31772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "8280d3a7ab31470c95fcb5f46f332595"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "9e6577a9116240960f39c4ce5e0af6ad"
server_key_iv: "fdff831f8b605d3f22625a164eaceb14"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.699965 31772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:23.703122 31801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.704298 31772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:23.704442 31772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "8280d3a7ab31470c95fcb5f46f332595"
format_stamp: "Formatted at 2026-08-10 18:48:23 on dist-test-slave-6zbq"
server_key: "9e6577a9116240960f39c4ce5e0af6ad"
server_key_iv: "fdff831f8b605d3f22625a164eaceb14"
server_key_version: "encryptionkey@0"
I20260810 18:48:23.704617 31772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:23.716696 31772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:23.720135 31772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:23.720381 31772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:23.721299 31772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:23.722503 31772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:23.722581 31772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.722695 31772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:23.722748 31772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:23.759088 31772 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:36777
I20260810 18:48:23.759138 31991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:36777 every 8 connection(s)
I20260810 18:48:23.760263 31772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:23.761269 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31772
I20260810 18:48:23.761463 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:23.762075 30395 external_mini_cluster.cc:1428] Setting key b44f5d833b486abc2513eee47420dc87
Aug 10 18:48:23 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387703, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:23.779840 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:23.763028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:40553 (local address 127.29.174.254:43701)
0810 18:48:23.763210 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:23.763216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:23.764396 (+ 1180us) server_negotiation.cc:407] Connection header received
0810 18:48:23.766243 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:23.766248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:23.766330 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:23.766457 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:23.767959 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.768781 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.770395 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.770790 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.770902 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:23.774425 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:23.774430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:23.774434 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:23.774466 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:23.776894 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.777596 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.777601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.777603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.777730 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:23.778142 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:23.778146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:23.778148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:23.778458 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:23.778586 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:23.779274 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 18:48:23.779500 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
I20260810 18:48:23.780779 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:23.763498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43701 (local address 127.29.174.194:40553)
0810 18:48:23.764254 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:23.764290 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:23.765991 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:23.766712 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:23.766721 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:23.767244 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:23.767798 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:23.767815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.768918 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.768922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:23.770234 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:23.770247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:23.771003 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:23.771010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:23.771255 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:23.771866 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:23.771888 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:23.774201 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:23.777040 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:23.777049 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:23.777066 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:23.777480 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:23.777847 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:23.777851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:23.777853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:23.778033 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:23.778591 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:23.778599 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:23.778922 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 18:48:23.779154 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 18:48:23.779658 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":318,"threads_started":1}
I20260810 18:48:23.782133 31992 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43701
I20260810 18:48:23.782467 31992 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
I20260810 18:48:23.783190 31992 heartbeater.cc:511] Master 127.29.174.254:43701 requested a full tablet report, sending...
I20260810 18:48:23.784639 31435 ts_manager.cc:194] Registered new tserver with Master: 8280d3a7ab31470c95fcb5f46f332595 (127.29.174.194:36777)
I20260810 18:48:23.785399 31435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:40553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:23.805117 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:24.118149 32004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:24.118433 32004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:24.118534 32004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:24.125470 32004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:24.125561 32004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:24.125676 32004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:24.134920 32004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:24.136003 32004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:24.137169 32004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:24.149296 32013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.149318 32015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:24.149328 32019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:24.150102 32004 server_base.cc:1034] running on GCE node
I20260810 18:48:24.150674 32004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:24.151556 32004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:24.152737 32004 hybrid_clock.cc:648] HybridClock initialized: now 1786387704152687 us; error 44 us; skew 500 ppm
I20260810 18:48:24.153034 32004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:24.155979 32004 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:24.157471 32004 webserver.cc:460] Webserver started at http://127.29.174.195:36717/ using document root <none> and password file <none>
I20260810 18:48:24.158056 32004 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:24.158154 32004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:24.158463 32004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:24.161872 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "b0fe350a37cc41d6b5665405e88b055f"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-6zbq"
server_key: "552833158a178c6e82d6549d2ab4a9ef"
server_key_iv: "63ea1d71651b85d3ae8247b1bfed1a83"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.162744 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "b0fe350a37cc41d6b5665405e88b055f"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-6zbq"
server_key: "552833158a178c6e82d6549d2ab4a9ef"
server_key_iv: "63ea1d71651b85d3ae8247b1bfed1a83"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.168910 32004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:48:24.173141 32034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.174453 32004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:24.174634 32004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "b0fe350a37cc41d6b5665405e88b055f"
format_stamp: "Formatted at 2026-08-10 18:48:24 on dist-test-slave-6zbq"
server_key: "552833158a178c6e82d6549d2ab4a9ef"
server_key_iv: "63ea1d71651b85d3ae8247b1bfed1a83"
server_key_version: "encryptionkey@0"
I20260810 18:48:24.174827 32004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:24.207690 32004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:24.212636 32004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:24.213057 32004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:24.214172 32004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:24.216373 32004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:24.216531 32004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.216740 32004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:24.216830 32004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:24.289038 32004 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:40649
I20260810 18:48:24.289229 32210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:40649 every 8 connection(s)
I20260810 18:48:24.290643 32004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:24.293558 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32004
I20260810 18:48:24.293756 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:24.294235 30395 external_mini_cluster.cc:1428] Setting key 7f02193fa03da644a8fc7eb7009e83c5
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:24.318269 31760 heartbeater.cc:503] Master 127.29.174.254:43701 was elected leader, sending a full tablet report...
I20260810 18:48:24.319424 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.294516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:41015 (local address 127.29.174.254:43701)
0810 18:48:24.295060 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.295067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.299499 (+ 4432us) server_negotiation.cc:407] Connection header received
0810 18:48:24.301896 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.301900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.301961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.302043 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.303952 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.305112 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.307064 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.307789 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.307929 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.312450 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.312455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.312458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.312489 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.315936 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.316690 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.316695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.316699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.316871 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.317364 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.317368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.317370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.317678 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.317807 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.318696 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.318887 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":290,"threads_started":1}
I20260810 18:48:24.320119 32219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.294387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43701 (local address 127.29.174.195:41015)
0810 18:48:24.299345 (+ 4958us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.299388 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.301689 (+ 2301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.302280 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.302287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.303003 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:24.303787 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.303806 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.305297 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.305301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.306933 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.306946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.308122 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.308133 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.308477 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.309298 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:24.309326 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:24.312231 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:24.316060 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.316069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.316083 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.316565 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.316975 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:24.316980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:24.316984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:24.317249 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:24.317801 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:24.317808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:24.318223 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.318549 (+ 326us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.318977 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4649,"thread_start_us":4470,"threads_started":1}
I20260810 18:48:24.321568 32212 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43701
I20260810 18:48:24.321907 32212 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:24.322831 32212 heartbeater.cc:511] Master 127.29.174.254:43701 requested a full tablet report, sending...
I20260810 18:48:24.324254 31435 ts_manager.cc:194] Registered new tserver with Master: b0fe350a37cc41d6b5665405e88b055f (127.29.174.195:40649)
I20260810 18:48:24.325186 31435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:41015
I20260810 18:48:24.337208 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:24.362241 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.347003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.29.174.254:43701)
0810 18:48:24.347178 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.347182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.348095 (+ 913us) server_negotiation.cc:407] Connection header received
0810 18:48:24.348246 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.348250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.348319 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.348396 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.350739 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.351688 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.353091 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.353622 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.353755 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.356294 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.356300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.356303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.356336 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.359578 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.360236 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.360240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.360243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.360398 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.360731 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.360734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.360737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.361203 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.361428 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.361783 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.361962 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:24.370678 31435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:24.381227 31435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:24.428686 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.398264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:40649 (local address 127.0.0.1:43452)
0810 18:48:24.411302 (+ 13038us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.411339 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.411574 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.419579 (+ 8005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.419588 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.419669 (+ 81us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:24.420385 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.420400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.422653 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.422658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.425273 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.425290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.427469 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.427480 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.427759 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.427814 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.427944 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.428148 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12751,"mutex_wait_us":23,"thread_start_us":12527,"threads_started":1}
I20260810 18:48:24.428720 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.398527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.29.174.195:40649)
0810 18:48:24.418274 (+ 19747us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.418283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.418322 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:48:24.418451 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.418458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.419258 (+ 800us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.420272 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:24.420755 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.422520 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.425476 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.427188 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.427376 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.427551 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.428001 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.428221 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19474,"thread_start_us":19174,"threads_started":1}
I20260810 18:48:24.430488 32240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.398264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:33523 (local address 127.0.0.1:46020)
0810 18:48:24.416671 (+ 18407us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.416689 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.416954 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.420837 (+ 3883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.420842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.420870 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:24.422027 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.422042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.424430 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.424435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.426976 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.426988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.429256 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.429264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.429507 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.429540 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.429634 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.429758 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18238,"mutex_wait_us":191,"thread_start_us":18629,"threads_started":1}
I20260810 18:48:24.431598 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.398636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.29.174.193:33523)
0810 18:48:24.420245 (+ 21609us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.420252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.420278 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:48:24.420405 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.420412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.420610 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.420862 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:24.422207 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.424288 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.427120 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.428975 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.429206 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.429373 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.430931 (+ 1558us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.431095 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21432,"thread_start_us":21261,"threads_started":1}
I20260810 18:48:24.431664 32243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.399155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:36777 (local address 127.0.0.1:53278)
0810 18:48:24.419327 (+ 20172us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.419344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.419489 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.420557 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.420561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.420584 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:24.421855 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.421868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.424835 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.424841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.427764 (+ 2923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.427777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.430335 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.430408 (+ 73us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.430685 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.430718 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.430796 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.430910 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20022,"mutex_wait_us":44,"thread_start_us":19743,"threads_started":1}
I20260810 18:48:24.432548 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.399736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.29.174.194:36777)
0810 18:48:24.418567 (+ 18831us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.418575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.419602 (+ 1027us) server_negotiation.cc:407] Connection header received
0810 18:48:24.419715 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.419721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.419969 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.420179 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:24.422107 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.424683 (+ 2576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.427908 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.429875 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.430041 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.431962 (+ 1921us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.432016 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.432188 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18647,"thread_start_us":18507,"threads_started":1}
I20260810 18:48:24.433779 32116 tablet_service.cc:1467] Processing CreateTablet for tablet c7575f8218544b4bad978ba80c95c751 (DEFAULT_TABLE table=test-table [id=b8209c5425c74494a916a3f478164c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:24.435406 32116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7575f8218544b4bad978ba80c95c751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:24.435930 31885 tablet_service.cc:1467] Processing CreateTablet for tablet c7575f8218544b4bad978ba80c95c751 (DEFAULT_TABLE table=test-table [id=b8209c5425c74494a916a3f478164c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:24.437592 31885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7575f8218544b4bad978ba80c95c751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:24.438259 31652 tablet_service.cc:1467] Processing CreateTablet for tablet c7575f8218544b4bad978ba80c95c751 (DEFAULT_TABLE table=test-table [id=b8209c5425c74494a916a3f478164c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:24.440097 31652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7575f8218544b4bad978ba80c95c751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:24.450048 32251 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Bootstrap starting.
I20260810 18:48:24.454787 32252 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Bootstrap starting.
I20260810 18:48:24.454923 32251 tablet_bootstrap.cc:654] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:24.456779 32251 log.cc:824] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:24.460070 32252 tablet_bootstrap.cc:654] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:24.460556 32253 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Bootstrap starting.
I20260810 18:48:24.460880 32251 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: No bootstrap required, opened a new log
I20260810 18:48:24.461261 32251 ts_tablet_manager.cc:1397] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260810 18:48:24.462198 32252 log.cc:824] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:24.464794 32253 tablet_bootstrap.cc:654] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:24.466044 32252 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: No bootstrap required, opened a new log
I20260810 18:48:24.466257 32253 log.cc:824] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:24.466394 32252 ts_tablet_manager.cc:1397] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260810 18:48:24.470043 32253 tablet_bootstrap.cc:492] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: No bootstrap required, opened a new log
I20260810 18:48:24.470428 32253 ts_tablet_manager.cc:1397] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260810 18:48:24.474460 32251 raft_consensus.cc:348] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.474911 32251 raft_consensus.cc:374] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:24.475071 32251 raft_consensus.cc:729] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fe350a37cc41d6b5665405e88b055f, State: Initialized, Role: FOLLOWER
I20260810 18:48:24.475802 32251 consensus_queue.cc:260] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.477401 32212 heartbeater.cc:503] Master 127.29.174.254:43701 was elected leader, sending a full tablet report...
I20260810 18:48:24.477834 32251 ts_tablet_manager.cc:1428] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260810 18:48:24.480165 32252 raft_consensus.cc:348] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.480664 32252 raft_consensus.cc:374] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:24.480839 32252 raft_consensus.cc:729] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8280d3a7ab31470c95fcb5f46f332595, State: Initialized, Role: FOLLOWER
I20260810 18:48:24.481429 32252 consensus_queue.cc:260] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.482827 31992 heartbeater.cc:503] Master 127.29.174.254:43701 was elected leader, sending a full tablet report...
I20260810 18:48:24.483577 32252 ts_tablet_manager.cc:1428] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260810 18:48:24.484239 32253 raft_consensus.cc:348] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.484881 32253 raft_consensus.cc:374] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:24.485080 32253 raft_consensus.cc:729] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42f91657d18a4294bbe81a3a8f3f9baf, State: Initialized, Role: FOLLOWER
I20260810 18:48:24.486503 32253 consensus_queue.cc:260] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.489924 32253 ts_tablet_manager.cc:1428] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Time spent starting tablet: real 0.019s user 0.006s sys 0.009s
I20260810 18:48:24.495910 32259 raft_consensus.cc:482] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:24.496302 32259 raft_consensus.cc:504] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.498802 32259 leader_election.cc:288] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42f91657d18a4294bbe81a3a8f3f9baf (127.29.174.193:33523), b0fe350a37cc41d6b5665405e88b055f (127.29.174.195:40649)
I20260810 18:48:24.509596 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.498924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:44773 (local address 127.29.174.193:33523)
0810 18:48:24.499171 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.499178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.500850 (+ 1672us) server_negotiation.cc:407] Connection header received
0810 18:48:24.501033 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.501039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.501128 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.501229 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:24.502414 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.503393 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.506722 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.508466 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.508657 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.508746 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.509234 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.509331 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:48:24.509918 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.499524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:46613 (local address 127.29.174.195:40649)
0810 18:48:24.499764 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.499770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.502080 (+ 2310us) server_negotiation.cc:407] Connection header received
0810 18:48:24.502296 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.502299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.502406 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.502507 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:24.503293 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.504034 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.507248 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.508743 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.508898 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.508964 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.509568 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.509701 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:48:24.509936 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.499374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:33523 (local address 127.29.174.194:44773)
0810 18:48:24.500745 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.500760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.500911 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.501272 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.501279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.501312 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:24.502249 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.502262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.503507 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.503510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.506464 (+ 2954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.506485 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.508770 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.508777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.508993 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.509036 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.509200 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.509337 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"mutex_wait_us":27,"thread_start_us":2041,"threads_started":1}
I20260810 18:48:24.510378 32263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.499509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:40649 (local address 127.29.174.194:46613)
0810 18:48:24.501972 (+ 2463us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.501990 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:24.502124 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:24.502513 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:24.502516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:24.502536 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:24.503088 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.503100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.504203 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.504208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.507071 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:24.507085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.509040 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.509047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:24.509393 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.509424 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:48:24.509522 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:48:24.509645 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2287,"mutex_wait_us":1,"thread_start_us":766,"threads_started":1}
I20260810 18:48:24.511390 31683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7575f8218544b4bad978ba80c95c751" candidate_uuid: "8280d3a7ab31470c95fcb5f46f332595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" is_pre_election: true
I20260810 18:48:24.511577 32143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7575f8218544b4bad978ba80c95c751" candidate_uuid: "8280d3a7ab31470c95fcb5f46f332595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe350a37cc41d6b5665405e88b055f" is_pre_election: true
W20260810 18:48:24.512259 31993 tablet.cc:2369] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:24.512444 31683 raft_consensus.cc:2454] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8280d3a7ab31470c95fcb5f46f332595 in term 0.
I20260810 18:48:24.513315 31802 leader_election.cc:302] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42f91657d18a4294bbe81a3a8f3f9baf, 8280d3a7ab31470c95fcb5f46f332595; no voters:
I20260810 18:48:24.513651 32259 raft_consensus.cc:2781] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:24.513827 32259 raft_consensus.cc:482] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:24.513947 32259 raft_consensus.cc:3037] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:24.514430 32143 raft_consensus.cc:2454] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8280d3a7ab31470c95fcb5f46f332595 in term 0.
I20260810 18:48:24.517064 32259 raft_consensus.cc:504] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.517794 32259 leader_election.cc:288] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [CANDIDATE]: Term 1 election: Requested vote from peers 42f91657d18a4294bbe81a3a8f3f9baf (127.29.174.193:33523), b0fe350a37cc41d6b5665405e88b055f (127.29.174.195:40649)
I20260810 18:48:24.518256 31683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7575f8218544b4bad978ba80c95c751" candidate_uuid: "8280d3a7ab31470c95fcb5f46f332595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42f91657d18a4294bbe81a3a8f3f9baf"
I20260810 18:48:24.518450 32143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7575f8218544b4bad978ba80c95c751" candidate_uuid: "8280d3a7ab31470c95fcb5f46f332595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fe350a37cc41d6b5665405e88b055f"
I20260810 18:48:24.518550 31683 raft_consensus.cc:3037] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:24.518697 32143 raft_consensus.cc:3037] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:24.522629 32143 raft_consensus.cc:2454] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8280d3a7ab31470c95fcb5f46f332595 in term 1.
I20260810 18:48:24.522819 31683 raft_consensus.cc:2454] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8280d3a7ab31470c95fcb5f46f332595 in term 1.
I20260810 18:48:24.523150 31812 leader_election.cc:302] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8280d3a7ab31470c95fcb5f46f332595, b0fe350a37cc41d6b5665405e88b055f; no voters:
I20260810 18:48:24.523536 32259 raft_consensus.cc:2781] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:24.524219 32259 raft_consensus.cc:686] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [term 1 LEADER]: Becoming Leader. State: Replica: 8280d3a7ab31470c95fcb5f46f332595, State: Running, Role: LEADER
I20260810 18:48:24.524789 32259 consensus_queue.cc:237] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } }
I20260810 18:48:24.531251 31435 catalog_manager.cc:5495] T c7575f8218544b4bad978ba80c95c751 P 8280d3a7ab31470c95fcb5f46f332595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8280d3a7ab31470c95fcb5f46f332595 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "8280d3a7ab31470c95fcb5f46f332595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f91657d18a4294bbe81a3a8f3f9baf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 33523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0fe350a37cc41d6b5665405e88b055f" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8280d3a7ab31470c95fcb5f46f332595" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36777 } health_report { overall_health: HEALTHY } } }
W20260810 18:48:24.538080 31761 tablet.cc:2369] T c7575f8218544b4bad978ba80c95c751 P 42f91657d18a4294bbe81a3a8f3f9baf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:24.542793 32214 tablet.cc:2369] T c7575f8218544b4bad978ba80c95c751 P b0fe350a37cc41d6b5665405e88b055f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:24.586102 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.566007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.29.174.193:33523)
0810 18:48:24.566181 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.566186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.567484 (+ 1298us) server_negotiation.cc:407] Connection header received
0810 18:48:24.567698 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.567702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.567804 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.567894 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.570500 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.571621 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.573373 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.574053 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.574174 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.579159 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.579171 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.579187 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.579219 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.582991 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.583755 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.583762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.583764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.583965 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.584410 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.584414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.584416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.584880 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.585209 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.585519 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.585738 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:24.644622 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.627989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.29.174.193:33523)
0810 18:48:24.628173 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.628180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.628819 (+ 639us) server_negotiation.cc:407] Connection header received
0810 18:48:24.629010 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.629014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.629075 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.629155 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.631291 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.632063 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.633536 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.634139 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.634251 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.638694 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.638698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.638702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.638732 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.642095 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.642672 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.642676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.642678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.642824 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.643240 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.643243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.643245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.643565 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.643831 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.643929 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.644115 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:24.696043 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.675366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.29.174.193:33523)
0810 18:48:24.675545 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.675552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.676094 (+ 542us) server_negotiation.cc:407] Connection header received
0810 18:48:24.676251 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.676256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.676317 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.676399 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.678387 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.679147 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.680731 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.681296 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.681407 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.686893 (+ 5486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.686899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.686903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.686944 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.692073 (+ 5129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.692931 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.692937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.692941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.693203 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.693646 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.693652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.693654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.694321 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.694876 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.695168 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.695537 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 18:48:24.698294 31615 tablet_service.cc:2908] Rejecting scan request for tablet c7575f8218544b4bad978ba80c95c751: Uninitialized: clean time has not yet been initialized
I20260810 18:48:24.702172 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31531
I20260810 18:48:24.716668 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.703054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.29.174.193:33523)
0810 18:48:24.703206 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.703210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.703673 (+ 463us) server_negotiation.cc:407] Connection header received
0810 18:48:24.703806 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.703809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.703880 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.703950 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.705425 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.706071 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.707462 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.707920 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.708018 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.710520 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.710527 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.710532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.710569 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.713982 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.714754 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.714758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.714761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.714925 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.715247 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.715250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.715252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.715562 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.716011 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.716191 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.716363 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:48:24.717259 31753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:24.827265 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31531
I20260810 18:48:24.850538 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31772
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:24.865357 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.851383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53280 (local address 127.29.174.194:36777)
0810 18:48:24.851587 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.851592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.851613 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:48:24.851725 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.851731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.851800 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.851885 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.853555 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.854270 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.855697 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.856092 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.856212 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.859898 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.859910 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.859925 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.859961 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.862487 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.863092 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.863099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.863102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.863363 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.863711 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.863717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.863721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.864111 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.864430 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.864668 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.864929 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:24.866039 31986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:24.953480 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31772
I20260810 18:48:24.953586 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.953080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:33523 (local address 127.29.174.194:49205)
0810 18:48:24.953260 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:24.953441 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:33523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:48:24.973918 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32004
Aug 10 18:48:24 dist-test-slave-6zbq krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:24.987851 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:24.974749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.29.174.195:40649)
0810 18:48:24.974932 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:24.974937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:24.974958 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:48:24.975054 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:24.975059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:24.975123 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:24.975207 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:24.976450 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.977182 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.978653 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:24.979077 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:24.979186 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:24.983052 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:24.983063 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:24.983078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:24.983114 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:24.985339 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.985916 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.985924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.985926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.986052 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:24.986427 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:24.986432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:24.986436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:24.986771 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:24.987052 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:24.987323 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 18:48:24.987526 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:48:24.988426 32204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:25.072937 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32004
I20260810 18:48:25.092731 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31392
I20260810 18:48:25.093796 31483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:25.204919 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31392
2026-08-10T18:48:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4297 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:25 dist-test-slave-6zbq krb5kdc[32297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:25 dist-test-slave-6zbq krb5kdc[32297](info): set up 2 sockets
Aug 10 18:48:25 dist-test-slave-6zbq krb5kdc[32297](info): commencing operation
krb5kdc: starting...
W20260810 18:48:26.396760 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:26 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:26.575102 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46057
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:46057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:26.879381 32316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:26.879642 32316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:26.879756 32316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:26.886662 32316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:26.886762 32316 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:48:26.886829 32316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:26.886910 32316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:26.886976 32316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:26.887056 32316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:26.896358 32316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:46057
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:26.897411 32316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:26.898646 32316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:26.909219 32328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:26.909255 32329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:26.909403 32331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:26.910254 32316 server_base.cc:1034] running on GCE node
I20260810 18:48:26.910971 32316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:26.912164 32316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:26.913403 32316 hybrid_clock.cc:648] HybridClock initialized: now 1786387706913350 us; error 27 us; skew 500 ppm
I20260810 18:48:26.913713 32316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:26 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:26.916608 32316 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:26.917977 32316 webserver.cc:460] Webserver started at http://127.29.174.254:38389/ using document root <none> and password file <none>
I20260810 18:48:26.918643 32316 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:26.918736 32316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:26.919034 32316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:26.921833 32316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "3629473b56984372a60edcd0c8726938"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-6zbq"
server_key: "44e04fc7ab543441168ca1bca8602c28"
server_key_iv: "a82b0c0b92d83854657b347f8ddb1a9b"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.922605 32316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "3629473b56984372a60edcd0c8726938"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-6zbq"
server_key: "44e04fc7ab543441168ca1bca8602c28"
server_key_iv: "a82b0c0b92d83854657b347f8ddb1a9b"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.927301 32316 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:48:26.930390 32340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:26.931511 32316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:26.931659 32316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "3629473b56984372a60edcd0c8726938"
format_stamp: "Formatted at 2026-08-10 18:48:26 on dist-test-slave-6zbq"
server_key: "44e04fc7ab543441168ca1bca8602c28"
server_key_iv: "a82b0c0b92d83854657b347f8ddb1a9b"
server_key_version: "encryptionkey@0"
I20260810 18:48:26.931838 32316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:26.946796 32316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:26.950183 32316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:26.950461 32316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:26.969489 32316 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:46057
I20260810 18:48:26.969508 32455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:46057 every 8 connection(s)
I20260810 18:48:26.970710 32316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:26.974072 32456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:26.980597 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32316
I20260810 18:48:26.980798 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:26.981475 30395 external_mini_cluster.cc:1428] Setting key 6eca65ed817e1e6b3ca68b96824a0602
I20260810 18:48:26.985925 32456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Bootstrap starting.
I20260810 18:48:26.989440 32456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:26.990485 32456 log.cc:824] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:26.993227 32456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: No bootstrap required, opened a new log
Aug 10 18:48:26 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:27.001099 32456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } }
I20260810 18:48:27.001451 32456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:27.001539 32456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3629473b56984372a60edcd0c8726938, State: Initialized, Role: FOLLOWER
I20260810 18:48:27.002102 32456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } }
I20260810 18:48:27.002363 32456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:27.002533 32456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:27.002703 32456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:27.003955 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:26.983961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.29.174.254:46057)
0810 18:48:26.984743 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:26.984756 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:26.984795 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:48:26.986549 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:26.986574 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:26.986929 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:26.987237 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:26.988941 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.990267 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.991729 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:26.992362 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:26.992481 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:26.997058 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:26.997066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:26.997087 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:26.997122 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:27.000047 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.000676 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.000682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.000688 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.000850 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.001219 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.001223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.001225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.001813 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:27.002105 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:27.002559 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 18:48:27.002951 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":357,"threads_started":1}
I20260810 18:48:27.005242 32456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } }
I20260810 18:48:27.005707 32456 leader_election.cc:302] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3629473b56984372a60edcd0c8726938; no voters:
I20260810 18:48:27.006309 32456 leader_election.cc:288] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:27.006461 32464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:27.007124 32464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [term 1 LEADER]: Becoming Leader. State: Replica: 3629473b56984372a60edcd0c8726938, State: Running, Role: LEADER
I20260810 18:48:27.007642 32464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } }
I20260810 18:48:27.008527 32456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:27.010429 32465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3629473b56984372a60edcd0c8726938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } } }
I20260810 18:48:27.010505 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3629473b56984372a60edcd0c8726938. Latest consensus state: current_term: 1 leader_uuid: "3629473b56984372a60edcd0c8726938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3629473b56984372a60edcd0c8726938" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46057 } } }
I20260810 18:48:27.010823 32465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:27.010839 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:27.011680 32475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:27.017196 32475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:27.026726 32475 catalog_manager.cc:1347] Generated new cluster ID: dc736f3d608348198e690979ba4b70ba
I20260810 18:48:27.026818 32475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:27.042691 32475 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:27.043809 32475 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:27.056861 32475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Generated new TSK 0
I20260810 18:48:27.057642 32475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:27.113472 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46057
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:27.436303 32498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:27.436545 32498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:27.436640 32498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:27.443504 32498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:27.443593 32498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:27.443678 32498 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:27.443763 32498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:27.452863 32498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:27.454051 32498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:27.455294 32498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:27.467280 32510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:27.467440 32509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:27.467273 32512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:27.468037 32498 server_base.cc:1034] running on GCE node
I20260810 18:48:27.468544 32498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:27.469317 32498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:27.470487 32498 hybrid_clock.cc:648] HybridClock initialized: now 1786387707470413 us; error 42 us; skew 500 ppm
I20260810 18:48:27.470755 32498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:27 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387707, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:27.473618 32498 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:27.474925 32498 webserver.cc:460] Webserver started at http://127.29.174.193:43997/ using document root <none> and password file <none>
I20260810 18:48:27.475538 32498 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:27.475634 32498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:27.475930 32498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:27.479043 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "0389d4eae3394450a07764ac36867a91"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "c36dab0c031d43534df6cd8a522527f8"
server_key_iv: "dab83e2e9da9770c74650758a10700f5"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.479960 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "0389d4eae3394450a07764ac36867a91"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "c36dab0c031d43534df6cd8a522527f8"
server_key_iv: "dab83e2e9da9770c74650758a10700f5"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.485137 32498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:48:27.488409 32523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:27.489591 32498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:27.489741 32498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "0389d4eae3394450a07764ac36867a91"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "c36dab0c031d43534df6cd8a522527f8"
server_key_iv: "dab83e2e9da9770c74650758a10700f5"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.489929 32498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:27.505443 32498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:27.509445 32498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:27.509673 32498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:27.512296 32498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:27.512380 32498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:27.512499 32498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:27.512558 32498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:27 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387707, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:27.537920 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:27.512890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:52215 (local address 127.29.174.254:46057)
0810 18:48:27.514690 (+ 1800us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:27.514697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:27.514827 (+ 130us) server_negotiation.cc:407] Connection header received
0810 18:48:27.518298 (+ 3471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:27.518305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:27.518438 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:27.518570 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:27.520225 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.521410 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.524358 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.525013 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.525139 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:27.530702 (+ 5563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:27.530709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:27.530713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:27.530761 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:27.534172 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.535170 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.535176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.535180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.535334 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.535861 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.535866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.535870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.536266 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:27.536428 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:27.537338 (+ 910us) server_negotiation.cc:299] Negotiation successful
0810 18:48:27.537555 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1679,"thread_start_us":1589,"threads_started":1}
I20260810 18:48:27.539425 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:27.513621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.193:52215)
0810 18:48:27.514658 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:27.514710 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:27.517847 (+ 3137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:27.518852 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:27.518862 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:27.519418 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:27.520038 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:27.520058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.521628 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.521635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:27.524171 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:27.524190 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.525254 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.525260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:27.525466 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:27.526646 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:27.526695 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:27.530328 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:27.534325 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:27.534359 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:27.534375 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:27.535027 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:27.535467 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:27.535472 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:27.535475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:27.535737 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:27.536510 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:27.536519 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:27.536950 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 18:48:27.537231 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 18:48:27.537810 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":386,"threads_started":1}
I20260810 18:48:27.558035 32498 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35739
I20260810 18:48:27.558053 32727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35739 every 8 connection(s)
I20260810 18:48:27.559628 32498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:27.560004 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:27.545271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:43767 (local address 127.29.174.254:46057)
0810 18:48:27.545467 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:27.545472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:27.546229 (+ 757us) server_negotiation.cc:407] Connection header received
0810 18:48:27.546419 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:27.546425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:27.546500 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:27.546608 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:27.547962 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.548666 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.550841 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.551348 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.551464 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:27.553771 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:27.553776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:27.553779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:27.553808 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:27.556949 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.557932 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.557937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.557941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.558110 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:27.558539 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:27.558543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:27.558545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:27.558906 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:27.559076 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:27.559551 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 18:48:27.559763 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:27.560189 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:27.545327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.193:43767)
0810 18:48:27.546130 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:27.546147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:27.546286 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:27.546660 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:27.546666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:27.547411 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:27.547817 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:27.547831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.548847 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.548852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:27.550701 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:27.550720 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:27.551693 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:27.551702 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:27.551882 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:27.552537 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:27.552562 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:27.553584 (+ 1022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:27.557135 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:27.557142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:27.557147 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:27.557750 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:27.558251 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:27.558257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:27.558260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:27.558432 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:27.559068 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:27.559073 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:27.559298 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 18:48:27.559445 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:48:27.559757 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":485,"threads_started":1}
I20260810 18:48:27.560918 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32498
I20260810 18:48:27.561131 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:27.561862 30395 external_mini_cluster.cc:1428] Setting key e94781262937697967dce7a0780f0dd2
I20260810 18:48:27.563746 32728 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46057
I20260810 18:48:27.564529 32728 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:27.565259 32728 heartbeater.cc:511] Master 127.29.174.254:46057 requested a full tablet report, sending...
I20260810 18:48:27.567301 32373 ts_manager.cc:194] Registered new tserver with Master: 0389d4eae3394450a07764ac36867a91 (127.29.174.193:35739)
I20260810 18:48:27.568683 32373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:52215
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:27.602856 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46057
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:27.919050 32735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:27.919307 32735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:27.919443 32735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:27.926465 32735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:27.926566 32735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:27.926698 32735 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:27.926818 32735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:27.936749 32735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:27.937983 32735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:27.939260 32735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:27.952280 32749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:27.952229 32745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:27.952255 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:27.952977 32735 server_base.cc:1034] running on GCE node
I20260810 18:48:27.953567 32735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:27.954520 32735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:27.955757 32735 hybrid_clock.cc:648] HybridClock initialized: now 1786387707955692 us; error 33 us; skew 500 ppm
I20260810 18:48:27.956054 32735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:27 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387707, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:27.962088 32735 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:27.963413 32735 webserver.cc:460] Webserver started at http://127.29.174.194:45697/ using document root <none> and password file <none>
I20260810 18:48:27.964016 32735 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:27.964098 32735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:27.964363 32735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:27.967141 32735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "1b4b7e3649fb469f9c686e727e9c71b1"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "1c9912aa176098546165e3eb6fc2a83a"
server_key_iv: "b065bfc4c2956ec4005f23d7871ec410"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.967849 32735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "1b4b7e3649fb469f9c686e727e9c71b1"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "1c9912aa176098546165e3eb6fc2a83a"
server_key_iv: "b065bfc4c2956ec4005f23d7871ec410"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.973086 32735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 18:48:27.976699 32756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:27.978015 32735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:27.978165 32735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "1b4b7e3649fb469f9c686e727e9c71b1"
format_stamp: "Formatted at 2026-08-10 18:48:27 on dist-test-slave-6zbq"
server_key: "1c9912aa176098546165e3eb6fc2a83a"
server_key_iv: "b065bfc4c2956ec4005f23d7871ec410"
server_key_version: "encryptionkey@0"
I20260810 18:48:27.978432 32735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:27.993782 32735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:27.998000 32735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:27.998247 32735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:28.001343 32735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:28.001430 32735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:28.001567 32735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:28.001690 32735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 18:48:28 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387707, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:28.023336 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.001876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:58715 (local address 127.29.174.254:46057)
0810 18:48:28.002088 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.002096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.003463 (+ 1367us) server_negotiation.cc:407] Connection header received
0810 18:48:28.005479 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.005484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.005566 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.005653 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.007835 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.008861 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.011685 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.012261 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.012380 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.016399 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.016404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.016407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.016440 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.019357 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.020309 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.020315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.020319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.020506 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.021176 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.021183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.021186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.021575 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.021753 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.022675 (+ 922us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.022937 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:48:28.024124 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.002465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.194:58715)
0810 18:48:28.003297 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.003345 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.005187 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.005903 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.005911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.006776 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:28.007641 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.007672 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.009034 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.009039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.011513 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.011529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.012521 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.012529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.012814 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.013491 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:28.013522 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:28.016191 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:28.019488 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.019497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.019515 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.020157 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.020657 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.020662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.020665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.020949 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.021713 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:28.021721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:28.022196 (+ 475us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.022534 (+ 338us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.022979 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":338,"threads_started":1}
I20260810 18:48:28.043076 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.030720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:42177 (local address 127.29.174.254:46057)
0810 18:48:28.030879 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.030885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.031444 (+ 559us) server_negotiation.cc:407] Connection header received
0810 18:48:28.031606 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.031612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.031686 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.031765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.032653 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.033362 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.034673 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.035254 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.035379 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.037564 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.037569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.037572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.037600 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.040430 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.041221 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.041224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.041227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.041359 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.041698 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.041701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.041703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.042046 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.042152 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.042661 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.042865 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:28.043377 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.030786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.194:42177)
0810 18:48:28.031343 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.031362 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.031488 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.031816 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.031820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.032244 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:28.032518 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.032526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.033498 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.033502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.034539 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.034558 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.035436 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.035446 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.035631 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.036328 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:28.036353 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:28.037391 (+ 1038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:28.040551 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.040557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.040562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.041111 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.041450 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.041455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.041459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.041609 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.042146 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:28.042151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:28.042397 (+ 246us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.042564 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.042906 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":299,"threads_started":1}
I20260810 18:48:28.057101 32735 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:42659
I20260810 18:48:28.057240 489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:42659 every 8 connection(s)
I20260810 18:48:28.058283 32735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:28.060201 490 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46057
I20260810 18:48:28.060391 490 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:28.061017 490 heartbeater.cc:511] Master 127.29.174.254:46057 requested a full tablet report, sending...
I20260810 18:48:28.061527 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32735
I20260810 18:48:28.061704 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:28.062227 30395 external_mini_cluster.cc:1428] Setting key 36b338803d4ab27e4b4fc9c145e88210
I20260810 18:48:28.062441 32373 ts_manager.cc:194] Registered new tserver with Master: 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659)
I20260810 18:48:28.063740 32373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:58715
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:28.105448 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46057
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:28.416932 500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:28.417196 500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:28.417326 500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:28.424573 500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:28.424697 500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:28.424819 500 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:28.424927 500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:28.435045 500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:28.436802 500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:28.438360 500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:28.459618 511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:28.459651 513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:28.459681 515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:28.460242 500 server_base.cc:1034] running on GCE node
I20260810 18:48:28.460896 500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:28.461819 500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:28.463049 500 hybrid_clock.cc:648] HybridClock initialized: now 1786387708462975 us; error 39 us; skew 500 ppm
I20260810 18:48:28.463358 500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:28 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:28.466583 500 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:28.467970 500 webserver.cc:460] Webserver started at http://127.29.174.195:42683/ using document root <none> and password file <none>
I20260810 18:48:28.468636 500 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:28.468734 500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:28.469033 500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:28.471969 500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "e783debcbbd34fd98af6dcf8294c47fb"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-6zbq"
server_key: "457ba5c21728c0e9f6af961518727160"
server_key_iv: "9128eeb1dc1e280bb9d9f6b17190e17b"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.472748 500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "e783debcbbd34fd98af6dcf8294c47fb"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-6zbq"
server_key: "457ba5c21728c0e9f6af961518727160"
server_key_iv: "9128eeb1dc1e280bb9d9f6b17190e17b"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.477779 500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:48:28.481011 529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:28.482511 500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:48:28.482656 500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "e783debcbbd34fd98af6dcf8294c47fb"
format_stamp: "Formatted at 2026-08-10 18:48:28 on dist-test-slave-6zbq"
server_key: "457ba5c21728c0e9f6af961518727160"
server_key_iv: "9128eeb1dc1e280bb9d9f6b17190e17b"
server_key_version: "encryptionkey@0"
I20260810 18:48:28.482877 500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:28.500866 500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:28.504890 500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:28.505139 500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:28.507835 500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:28.507973 500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 18:48:28.508109 500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:28.508186 500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:28 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:28.528931 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.508034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35527 (local address 127.29.174.254:46057)
0810 18:48:28.508219 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.508225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.509684 (+ 1459us) server_negotiation.cc:407] Connection header received
0810 18:48:28.511815 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.511820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.511904 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.511992 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.513874 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.514928 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.517325 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.517974 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.518123 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.521853 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.521861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.521867 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.521912 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.525453 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.526300 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.526305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.526309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.526514 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.527090 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.527094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.527096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.527476 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.527627 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.528487 (+ 860us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.528664 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:48:28.530550 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.508619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.195:35527)
0810 18:48:28.509502 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.509557 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.511524 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.512231 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.512239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.512764 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:28.513630 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.513668 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.515121 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.515127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.517162 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.517183 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.518196 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.518205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.518514 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.519187 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:28.519211 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:28.521623 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:28.525654 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.525664 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.525680 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.526106 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.526651 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.526660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.526665 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.526957 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.527637 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:28.527646 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:28.528113 (+ 467us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.528431 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.528982 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":380,"threads_started":1}
I20260810 18:48:28.550827 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.536316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:54577 (local address 127.29.174.254:46057)
0810 18:48:28.536456 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.536460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.537058 (+ 598us) server_negotiation.cc:407] Connection header received
0810 18:48:28.537212 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.537218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.537281 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.537358 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.538218 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.539131 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.540355 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.540766 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.540874 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.542648 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.542653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.542656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.542688 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.547809 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.548590 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.548596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.548600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.548762 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.549285 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.549290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.549293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.549651 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.549773 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.550328 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.550552 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 18:48:28.551314 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.536417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.29.174.195:54577)
0810 18:48:28.536965 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.536980 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.537102 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.537393 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.537398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.537811 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:28.538085 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.538094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.539269 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.539273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.540223 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.540235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.540945 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.540952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.541130 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.541733 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:28.541754 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:28.542461 (+ 707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:28.547960 (+ 5499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.547969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.547973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.548466 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.548913 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:28.548920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:28.548924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:28.549119 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:28.549771 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:28.549777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:28.550024 (+ 247us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.550225 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.550671 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":310,"threads_started":1}
I20260810 18:48:28.567909 500 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37607
I20260810 18:48:28.567930 747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37607 every 8 connection(s)
I20260810 18:48:28.568960 500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:28.570914 748 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46057
I20260810 18:48:28.571098 748 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:28.571626 32728 heartbeater.cc:503] Master 127.29.174.254:46057 was elected leader, sending a full tablet report...
I20260810 18:48:28.571779 748 heartbeater.cc:511] Master 127.29.174.254:46057 requested a full tablet report, sending...
I20260810 18:48:28.573163 32371 ts_manager.cc:194] Registered new tserver with Master: e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607)
I20260810 18:48:28.573935 32371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:35527
I20260810 18:48:28.574899 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 500
I20260810 18:48:28.575090 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:28.575631 30395 external_mini_cluster.cc:1428] Setting key 6f518fe83d02eac3dc85bc3f32585b4a
I20260810 18:48:28.578251 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:28 dist-test-slave-6zbq krb5kdc[32297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:28 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:28.628976 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.616048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.29.174.254:46057)
0810 18:48:28.616258 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.616264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.616488 (+ 224us) server_negotiation.cc:407] Connection header received
0810 18:48:28.616633 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.616637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.616712 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.616795 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:28.618081 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.618887 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.620071 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.620450 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.620543 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.624288 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:28.624293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:28.624296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:28.624330 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:28.626793 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.627355 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.627359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.627361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.627476 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:28.627815 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:28.627818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:28.627820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:28.628078 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:28.628269 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.628513 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.628674 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:28.632803 32371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:28.639498 32371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:28.661492 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.652006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.29.174.194:42659)
0810 18:48:28.652699 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.652708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.653357 (+ 649us) server_negotiation.cc:407] Connection header received
0810 18:48:28.653526 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.653534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.653768 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.653945 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:28.654841 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.656280 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.658804 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.660097 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.660247 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.660418 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.661020 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.661203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":407,"threads_started":1}
I20260810 18:48:28.661573 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.651409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:42659 (local address 127.0.0.1:39400)
0810 18:48:28.653123 (+ 1714us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.653138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.653375 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.654158 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.654162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.654200 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:28.654659 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.654673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.656438 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.656444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.658603 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.658622 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.660583 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.660590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.660776 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.660823 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.660954 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.661141 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1592,"mutex_wait_us":35,"thread_start_us":1239,"threads_started":1}
I20260810 18:48:28.662626 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.651586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.29.174.195:37607)
0810 18:48:28.652464 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.652472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.653415 (+ 943us) server_negotiation.cc:407] Connection header received
0810 18:48:28.653521 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.653527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.653713 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.653910 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:28.655884 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.657251 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.659945 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.660896 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.661093 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.661240 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.662145 (+ 905us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.662294 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":557,"threads_started":1}
I20260810 18:48:28.662776 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.653940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35739 (local address 127.0.0.1:52110)
0810 18:48:28.655434 (+ 1494us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.655450 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.655556 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.656112 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.656116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.656138 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:28.656547 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.656558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.658369 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.658374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.659983 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.659997 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.662002 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.662007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.662151 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.662182 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.662260 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.662393 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"mutex_wait_us":4,"thread_start_us":1274,"threads_started":1}
I20260810 18:48:28.663055 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.651295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37607 (local address 127.0.0.1:43250)
0810 18:48:28.653021 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:28.653066 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:28.653243 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:28.653918 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:28.653926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:28.653956 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:28.654410 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.654423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.657387 (+ 2964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.657393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.659774 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:28.659790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.661382 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.661388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:28.661545 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.661581 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:48:28.661670 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:48:28.661786 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1450,"mutex_wait_us":18,"thread_start_us":1556,"threads_started":1}
I20260810 18:48:28.663439 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:28.654843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.29.174.193:35739)
0810 18:48:28.655725 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:28.655731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:28.655749 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:48:28.655814 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:28.655819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:28.655953 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:28.656108 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:28.656756 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.658023 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.660164 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:28.661538 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:28.661711 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:28.661874 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:28.663023 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 18:48:28.663155 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":594,"threads_started":1}
I20260810 18:48:28.665740 379 tablet_service.cc:1467] Processing CreateTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 (DEFAULT_TABLE table=test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:28.666239 32602 tablet_service.cc:1467] Processing CreateTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 (DEFAULT_TABLE table=test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:28.667380 379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258efed7f1f647d3b41f2cd5f1d6f067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:28.666939 620 tablet_service.cc:1467] Processing CreateTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 (DEFAULT_TABLE table=test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:28.667691 32602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258efed7f1f647d3b41f2cd5f1d6f067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:28.668457 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 258efed7f1f647d3b41f2cd5f1d6f067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:28.682174 781 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Bootstrap starting.
I20260810 18:48:28.686825 785 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Bootstrap starting.
I20260810 18:48:28.687943 784 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Bootstrap starting.
I20260810 18:48:28.687984 781 tablet_bootstrap.cc:654] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:28.690039 781 log.cc:824] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:28.691365 785 tablet_bootstrap.cc:654] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:28.692150 784 tablet_bootstrap.cc:654] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:28.692981 785 log.cc:824] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:28.694213 784 log.cc:824] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:28.696542 781 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: No bootstrap required, opened a new log
I20260810 18:48:28.696918 781 ts_tablet_manager.cc:1397] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20260810 18:48:28.700266 785 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: No bootstrap required, opened a new log
I20260810 18:48:28.700265 784 tablet_bootstrap.cc:492] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: No bootstrap required, opened a new log
I20260810 18:48:28.700654 784 ts_tablet_manager.cc:1397] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260810 18:48:28.700690 785 ts_tablet_manager.cc:1397] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260810 18:48:28.711035 781 raft_consensus.cc:348] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.711668 781 raft_consensus.cc:374] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:28.711428 784 raft_consensus.cc:348] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.711812 781 raft_consensus.cc:729] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0389d4eae3394450a07764ac36867a91, State: Initialized, Role: FOLLOWER
I20260810 18:48:28.711917 784 raft_consensus.cc:374] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:28.712028 784 raft_consensus.cc:729] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e783debcbbd34fd98af6dcf8294c47fb, State: Initialized, Role: FOLLOWER
I20260810 18:48:28.712658 781 consensus_queue.cc:260] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.712658 784 consensus_queue.cc:260] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.714900 785 raft_consensus.cc:348] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.715576 785 raft_consensus.cc:374] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:28.715783 781 ts_tablet_manager.cc:1428] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Time spent starting tablet: real 0.019s user 0.012s sys 0.005s
I20260810 18:48:28.716001 748 heartbeater.cc:503] Master 127.29.174.254:46057 was elected leader, sending a full tablet report...
I20260810 18:48:28.716624 784 ts_tablet_manager.cc:1428] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
I20260810 18:48:28.716805 785 raft_consensus.cc:729] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4b7e3649fb469f9c686e727e9c71b1, State: Initialized, Role: FOLLOWER
I20260810 18:48:28.717388 785 consensus_queue.cc:260] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:28.718703 490 heartbeater.cc:503] Master 127.29.174.254:46057 was elected leader, sending a full tablet report...
I20260810 18:48:28.719463 785 ts_tablet_manager.cc:1428] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent starting tablet: real 0.019s user 0.008s sys 0.009s
W20260810 18:48:28.810576 493 tablet.cc:2369] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:28.814469 32730 tablet.cc:2369] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:28.820717 751 tablet.cc:2369] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:29.088264 791 raft_consensus.cc:482] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:29.088742 791 raft_consensus.cc:504] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:29.091014 791 leader_election.cc:288] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607), 0389d4eae3394450a07764ac36867a91 (127.29.174.193:35739)
I20260810 18:48:29.101087 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.091413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:49021 (local address 127.29.174.195:37607)
0810 18:48:29.091631 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.091639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.092721 (+ 1082us) server_negotiation.cc:407] Connection header received
0810 18:48:29.093010 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.093017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.093107 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.093241 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:29.094321 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.095601 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.098480 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.100028 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.100171 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.100228 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.100712 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.100802 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:48:29.101251 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.091322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37607 (local address 127.29.174.194:49021)
0810 18:48:29.092605 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.092624 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.092798 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.093273 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.093278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.093310 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:29.094163 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.094177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.095783 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.095789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.098257 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.098270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.100357 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.100364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.100555 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.100595 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.100701 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.100837 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"thread_start_us":766,"threads_started":1}
I20260810 18:48:29.102360 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.092701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:55177 (local address 127.29.174.193:35739)
0810 18:48:29.092893 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.092899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.093948 (+ 1049us) server_negotiation.cc:407] Connection header received
0810 18:48:29.094162 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.094168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.094259 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.094362 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:29.095515 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.096584 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.099875 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.101226 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.101416 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.101488 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.101941 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.102065 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:48:29.102409 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.092613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35739 (local address 127.29.174.194:55177)
0810 18:48:29.093847 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.093866 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.094022 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.094398 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.094403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.094430 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:29.095303 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.095318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.096767 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.096771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.099682 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.099700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.101623 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.101631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.101784 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.101809 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.101881 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.101984 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":728,"threads_started":1}
I20260810 18:48:29.102581 664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258efed7f1f647d3b41f2cd5f1d6f067" candidate_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb" is_pre_election: true
I20260810 18:48:29.103251 664 raft_consensus.cc:2454] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4b7e3649fb469f9c686e727e9c71b1 in term 0.
I20260810 18:48:29.103938 32759 leader_election.cc:302] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4b7e3649fb469f9c686e727e9c71b1, e783debcbbd34fd98af6dcf8294c47fb; no voters:
I20260810 18:48:29.103724 32637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258efed7f1f647d3b41f2cd5f1d6f067" candidate_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0389d4eae3394450a07764ac36867a91" is_pre_election: true
I20260810 18:48:29.104342 791 raft_consensus.cc:2781] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:29.104472 791 raft_consensus.cc:482] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:29.104424 32637 raft_consensus.cc:2454] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4b7e3649fb469f9c686e727e9c71b1 in term 0.
I20260810 18:48:29.104566 791 raft_consensus.cc:3037] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.107529 791 raft_consensus.cc:504] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:29.108270 791 leader_election.cc:288] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [CANDIDATE]: Term 1 election: Requested vote from peers e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607), 0389d4eae3394450a07764ac36867a91 (127.29.174.193:35739)
I20260810 18:48:29.108711 664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258efed7f1f647d3b41f2cd5f1d6f067" candidate_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb"
I20260810 18:48:29.108759 32637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "258efed7f1f647d3b41f2cd5f1d6f067" candidate_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0389d4eae3394450a07764ac36867a91"
I20260810 18:48:29.108914 32637 raft_consensus.cc:3037] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.108960 664 raft_consensus.cc:3037] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.111639 32637 raft_consensus.cc:2454] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4b7e3649fb469f9c686e727e9c71b1 in term 1.
I20260810 18:48:29.111671 664 raft_consensus.cc:2454] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4b7e3649fb469f9c686e727e9c71b1 in term 1.
I20260810 18:48:29.112011 32759 leader_election.cc:302] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0389d4eae3394450a07764ac36867a91, 1b4b7e3649fb469f9c686e727e9c71b1; no voters:
I20260810 18:48:29.112341 791 raft_consensus.cc:2781] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:29.113111 791 raft_consensus.cc:686] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 LEADER]: Becoming Leader. State: Replica: 1b4b7e3649fb469f9c686e727e9c71b1, State: Running, Role: LEADER
I20260810 18:48:29.113720 791 consensus_queue.cc:237] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } }
I20260810 18:48:29.118597 32371 catalog_manager.cc:5495] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:29.237896 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.231901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.29.174.194:42659)
0810 18:48:29.232786 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.232790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.232808 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:48:29.233008 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.233013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.233092 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.233219 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:29.233623 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.234303 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.235884 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.236289 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.236380 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.236897 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.237417 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.237518 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.237600 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.237664 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":659,"threads_started":1}
I20260810 18:48:29.247226 664 raft_consensus.cc:1261] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Refusing update from remote peer 1b4b7e3649fb469f9c686e727e9c71b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.248421 799 consensus_queue.cc:1035] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 18:48:29.249095 32637 raft_consensus.cc:1261] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Refusing update from remote peer 1b4b7e3649fb469f9c686e727e9c71b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.249926 799 consensus_queue.cc:1035] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:29.262490 810 mvcc.cc:205] Tried to move back new op lower bound from 7317044057064411136 to 7317044056533954560. Current Snapshot: MvccSnapshot[applied={T|T < 7317044057064411136}]
I20260810 18:48:29.285017 32371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:29.285351 32371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:29.289395 32371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Sending DeleteTablet for 3 replicas of tablet 258efed7f1f647d3b41f2cd5f1d6f067
I20260810 18:48:29.290908 379 tablet_service.cc:1514] Processing DeleteTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:39400
I20260810 18:48:29.290925 32602 tablet_service.cc:1514] Processing DeleteTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:52110
I20260810 18:48:29.292346 620 tablet_service.cc:1514] Processing DeleteTablet for tablet 258efed7f1f647d3b41f2cd5f1d6f067 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:43250
I20260810 18:48:29.292351 32371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:29.293224 32371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:29.295509 830 tablet_replica.cc:330] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: stopping tablet replica
I20260810 18:48:29.296216 830 raft_consensus.cc:2229] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:29.296989 830 raft_consensus.cc:2258] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.301730 830 ts_tablet_manager.cc:1905] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.302096 378 tablet_service.cc:1467] Processing CreateTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 (DEFAULT_TABLE table=test-table [id=e01c43f834274ab99aae65374c2238b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:29.303061 378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eaf6a13d2446268a35c983f2491bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.305751 32601 tablet_service.cc:1467] Processing CreateTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 (DEFAULT_TABLE table=test-table [id=e01c43f834274ab99aae65374c2238b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:29.305831 620 tablet_service.cc:1467] Processing CreateTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 (DEFAULT_TABLE table=test-table [id=e01c43f834274ab99aae65374c2238b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:29.306380 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eaf6a13d2446268a35c983f2491bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.306582 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eaf6a13d2446268a35c983f2491bb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.312901 828 tablet_replica.cc:330] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: stopping tablet replica
I20260810 18:48:29.314728 828 raft_consensus.cc:2229] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:29.316466 828 raft_consensus.cc:2258] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.318981 828 ts_tablet_manager.cc:1905] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.319093 830 ts_tablet_manager.cc:1918] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:29.319342 830 log.cc:1196] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/258efed7f1f647d3b41f2cd5f1d6f067
I20260810 18:48:29.321885 830 ts_tablet_manager.cc:1939] T 258efed7f1f647d3b41f2cd5f1d6f067 P e783debcbbd34fd98af6dcf8294c47fb: Deleting consensus metadata
I20260810 18:48:29.324249 32347 catalog_manager.cc:4841] TS e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607): tablet 258efed7f1f647d3b41f2cd5f1d6f067 (table test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]) successfully deleted
I20260810 18:48:29.325476 832 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Bootstrap starting.
I20260810 18:48:29.329213 832 tablet_bootstrap.cc:654] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.329685 831 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Bootstrap starting.
I20260810 18:48:29.329973 828 ts_tablet_manager.cc:1918] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:29.330318 828 log.cc:1196] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/258efed7f1f647d3b41f2cd5f1d6f067
I20260810 18:48:29.331171 828 ts_tablet_manager.cc:1939] T 258efed7f1f647d3b41f2cd5f1d6f067 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting consensus metadata
I20260810 18:48:29.332051 829 tablet_replica.cc:330] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: stopping tablet replica
I20260810 18:48:29.332551 831 tablet_bootstrap.cc:654] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.332695 829 raft_consensus.cc:2229] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:29.333451 32342 catalog_manager.cc:4841] TS 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659): tablet 258efed7f1f647d3b41f2cd5f1d6f067 (table test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]) successfully deleted
I20260810 18:48:29.333689 829 raft_consensus.cc:2258] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.335211 833 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Bootstrap starting.
I20260810 18:48:29.336272 829 ts_tablet_manager.cc:1905] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.337059 831 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: No bootstrap required, opened a new log
I20260810 18:48:29.337301 831 ts_tablet_manager.cc:1397] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:48:29.338603 833 tablet_bootstrap.cc:654] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.338464 831 raft_consensus.cc:348] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.338953 831 raft_consensus.cc:374] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.339129 831 raft_consensus.cc:729] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e783debcbbd34fd98af6dcf8294c47fb, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.339787 832 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: No bootstrap required, opened a new log
I20260810 18:48:29.339526 831 consensus_queue.cc:260] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.339980 832 ts_tablet_manager.cc:1397] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.005s
I20260810 18:48:29.341109 831 ts_tablet_manager.cc:1428] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 18:48:29.341250 832 raft_consensus.cc:348] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.341563 832 raft_consensus.cc:374] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.341779 832 raft_consensus.cc:729] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0389d4eae3394450a07764ac36867a91, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.342232 832 consensus_queue.cc:260] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.343017 832 ts_tablet_manager.cc:1428] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 18:48:29.350493 829 ts_tablet_manager.cc:1918] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:29.350725 829 log.cc:1196] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/258efed7f1f647d3b41f2cd5f1d6f067
I20260810 18:48:29.351274 829 ts_tablet_manager.cc:1939] T 258efed7f1f647d3b41f2cd5f1d6f067 P 0389d4eae3394450a07764ac36867a91: Deleting consensus metadata
I20260810 18:48:29.352945 32347 catalog_manager.cc:4841] TS 0389d4eae3394450a07764ac36867a91 (127.29.174.193:35739): tablet 258efed7f1f647d3b41f2cd5f1d6f067 (table test-table [id=cd6ab8e266814cd1a51dbf0968eb00d8]) successfully deleted
I20260810 18:48:29.353417 833 tablet_bootstrap.cc:492] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: No bootstrap required, opened a new log
I20260810 18:48:29.353611 833 ts_tablet_manager.cc:1397] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260810 18:48:29.354969 833 raft_consensus.cc:348] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.355464 833 raft_consensus.cc:374] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.355813 833 raft_consensus.cc:729] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4b7e3649fb469f9c686e727e9c71b1, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.356796 833 consensus_queue.cc:260] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.357650 833 ts_tablet_manager.cc:1428] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 18:48:29.490602 809 raft_consensus.cc:482] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:29.490931 809 raft_consensus.cc:504] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.493062 809 leader_election.cc:288] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659), e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607)
I20260810 18:48:29.502900 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.493535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48017 (local address 127.29.174.195:37607)
0810 18:48:29.493699 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.493706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.495872 (+ 2166us) server_negotiation.cc:407] Connection header received
0810 18:48:29.496156 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.496160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.496243 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.496330 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:29.497148 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.498273 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.500555 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.501730 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.501867 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.501949 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.502367 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.502501 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:48:29.503175 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.493859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37607 (local address 127.29.174.193:48017)
0810 18:48:29.495757 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.495775 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.496014 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.496366 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.496371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.496398 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:29.496947 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.496961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.498426 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.498432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.500383 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.500396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.501979 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.501987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.502194 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.502225 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.502307 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.502460 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1717,"mutex_wait_us":86,"thread_start_us":791,"threads_started":1}
I20260810 18:48:29.504029 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eaf6a13d2446268a35c983f2491bb6" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb" is_pre_election: true
I20260810 18:48:29.504387 663 raft_consensus.cc:2454] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 0.
I20260810 18:48:29.504949 32528 leader_election.cc:302] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0389d4eae3394450a07764ac36867a91, e783debcbbd34fd98af6dcf8294c47fb; no voters:
I20260810 18:48:29.505416 809 raft_consensus.cc:2781] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:29.505610 809 raft_consensus.cc:482] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:29.505721 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.493461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:37517 (local address 127.29.174.194:42659)
0810 18:48:29.493657 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.493663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.495944 (+ 2281us) server_negotiation.cc:407] Connection header received
0810 18:48:29.496265 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.496270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.496345 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.496428 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:29.497631 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.499100 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.502392 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.504178 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.504385 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.504457 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.505187 (+ 730us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.505335 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:48:29.505774 809 raft_consensus.cc:3037] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.506556 843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.494452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:42659 (local address 127.29.174.193:37517)
0810 18:48:29.495849 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.495867 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.496063 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.496657 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.496662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.496695 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:29.497449 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.497464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.499319 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.499325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.502184 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.502210 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.504703 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.504710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.504983 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.505018 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.505914 (+ 896us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.506054 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"mutex_wait_us":6,"thread_start_us":791,"threads_started":1}
I20260810 18:48:29.509645 809 raft_consensus.cc:504] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.509711 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eaf6a13d2446268a35c983f2491bb6" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" is_pre_election: true
I20260810 18:48:29.510701 424 raft_consensus.cc:2454] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 0.
I20260810 18:48:29.511320 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eaf6a13d2446268a35c983f2491bb6" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb"
I20260810 18:48:29.511544 663 raft_consensus.cc:3037] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.511440 423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eaf6a13d2446268a35c983f2491bb6" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4b7e3649fb469f9c686e727e9c71b1"
I20260810 18:48:29.511888 423 raft_consensus.cc:3037] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.512403 809 leader_election.cc:288] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 election: Requested vote from peers 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659), e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607)
I20260810 18:48:29.514595 663 raft_consensus.cc:2454] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 1.
I20260810 18:48:29.514999 32528 leader_election.cc:302] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0389d4eae3394450a07764ac36867a91, e783debcbbd34fd98af6dcf8294c47fb; no voters:
I20260810 18:48:29.515277 809 raft_consensus.cc:2781] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:29.516130 809 raft_consensus.cc:686] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 1 LEADER]: Becoming Leader. State: Replica: 0389d4eae3394450a07764ac36867a91, State: Running, Role: LEADER
I20260810 18:48:29.516613 423 raft_consensus.cc:2454] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 1.
I20260810 18:48:29.516746 809 consensus_queue.cc:237] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.520853 32371 catalog_manager.cc:5495] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0389d4eae3394450a07764ac36867a91 (127.29.174.193). New cstate: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:48:29 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387706, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:29.613695 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.569824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.29.174.254:46057)
0810 18:48:29.595182 (+ 25358us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.595189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.595216 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:48:29.595278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.595283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.595362 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.595480 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.597569 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.598894 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.601046 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.601806 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.601926 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.606281 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.606287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.606290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.606318 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.609944 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.611050 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.611055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.611057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.611226 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.611789 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.611793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.611795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.612242 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.612343 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.612988 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.613364 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":25216,"thread_start_us":25837,"threads_started":1}
I20260810 18:48:29.613788 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.570015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46057 (local address 127.0.0.1:52750)
0810 18:48:29.591234 (+ 21219us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.591279 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.591459 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.595567 (+ 4108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.595572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.596855 (+ 1283us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:29.597366 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.597381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.599070 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.599075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.600890 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.600908 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.602025 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.602032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.602274 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.603049 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:29.603073 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:29.606031 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:29.610080 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.610095 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.610102 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.610939 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.611354 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:29.611359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:29.611363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:29.611667 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:29.612417 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:29.612425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:29.612645 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.612894 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.613198 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21008,"thread_start_us":20768,"threads_started":1}
I20260810 18:48:29.620132 32371 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:52750:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:48:29.621555 32371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:29.628790 32602 tablet_service.cc:1467] Processing CreateTablet for tablet 592a6a6141da438e84418f375739ddf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba6c349754084d1b93e87707a99f2b3e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:29.629169 378 tablet_service.cc:1467] Processing CreateTablet for tablet 592a6a6141da438e84418f375739ddf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba6c349754084d1b93e87707a99f2b3e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:29.629412 32602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592a6a6141da438e84418f375739ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.629434 620 tablet_service.cc:1467] Processing CreateTablet for tablet 592a6a6141da438e84418f375739ddf9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba6c349754084d1b93e87707a99f2b3e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:29.629776 378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592a6a6141da438e84418f375739ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.630039 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592a6a6141da438e84418f375739ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:29.638473 831 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: Bootstrap starting.
I20260810 18:48:29.639693 833 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: Bootstrap starting.
I20260810 18:48:29.640656 831 tablet_bootstrap.cc:654] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.642819 833 tablet_bootstrap.cc:654] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.644212 831 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: No bootstrap required, opened a new log
I20260810 18:48:29.644306 832 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: Bootstrap starting.
I20260810 18:48:29.644577 831 ts_tablet_manager.cc:1397] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260810 18:48:29.645603 831 raft_consensus.cc:348] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.645951 831 raft_consensus.cc:374] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.646034 831 raft_consensus.cc:729] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e783debcbbd34fd98af6dcf8294c47fb, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.646334 831 consensus_queue.cc:260] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.646581 832 tablet_bootstrap.cc:654] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:29.646927 808 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.647702 808 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.651273 831 ts_tablet_manager.cc:1428] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260810 18:48:29.657050 833 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: No bootstrap required, opened a new log
I20260810 18:48:29.657387 833 ts_tablet_manager.cc:1397] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.002s
I20260810 18:48:29.658545 833 raft_consensus.cc:348] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.658975 833 raft_consensus.cc:374] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.659140 833 raft_consensus.cc:729] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4b7e3649fb469f9c686e727e9c71b1, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.659593 832 tablet_bootstrap.cc:492] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: No bootstrap required, opened a new log
I20260810 18:48:29.659569 833 consensus_queue.cc:260] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.659853 832 ts_tablet_manager.cc:1397] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260810 18:48:29.660252 833 ts_tablet_manager.cc:1428] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:29.660190 791 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.660630 791 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.661721 832 raft_consensus.cc:348] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.662103 832 raft_consensus.cc:374] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:29.662308 832 raft_consensus.cc:729] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0389d4eae3394450a07764ac36867a91, State: Initialized, Role: FOLLOWER
I20260810 18:48:29.662668 832 consensus_queue.cc:260] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.663285 832 ts_tablet_manager.cc:1428] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:29.663218 809 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.663515 809 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.672855 809 raft_consensus.cc:482] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:29.673146 809 raft_consensus.cc:504] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.674018 809 leader_election.cc:288] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659), e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607)
I20260810 18:48:29.674587 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592a6a6141da438e84418f375739ddf9" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb" is_pre_election: true
I20260810 18:48:29.674564 423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592a6a6141da438e84418f375739ddf9" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" is_pre_election: true
I20260810 18:48:29.674851 423 raft_consensus.cc:2454] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 0.
I20260810 18:48:29.674904 663 raft_consensus.cc:2454] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 0.
I20260810 18:48:29.675395 32525 leader_election.cc:302] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0389d4eae3394450a07764ac36867a91, 1b4b7e3649fb469f9c686e727e9c71b1; no voters:
I20260810 18:48:29.675715 809 raft_consensus.cc:2781] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:29.675853 809 raft_consensus.cc:482] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:29.676141 809 raft_consensus.cc:3037] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.680043 809 raft_consensus.cc:504] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.681267 423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592a6a6141da438e84418f375739ddf9" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4b7e3649fb469f9c686e727e9c71b1"
I20260810 18:48:29.681360 663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592a6a6141da438e84418f375739ddf9" candidate_uuid: "0389d4eae3394450a07764ac36867a91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e783debcbbd34fd98af6dcf8294c47fb"
I20260810 18:48:29.681645 423 raft_consensus.cc:3037] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.681704 663 raft_consensus.cc:3037] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:29.681974 809 leader_election.cc:288] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 election: Requested vote from peers 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659), e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607)
I20260810 18:48:29.684206 663 raft_consensus.cc:2454] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 1.
I20260810 18:48:29.684590 32528 leader_election.cc:302] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0389d4eae3394450a07764ac36867a91, e783debcbbd34fd98af6dcf8294c47fb; no voters:
I20260810 18:48:29.684820 809 raft_consensus.cc:2781] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:29.684971 809 raft_consensus.cc:686] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [term 1 LEADER]: Becoming Leader. State: Replica: 0389d4eae3394450a07764ac36867a91, State: Running, Role: LEADER
I20260810 18:48:29.685268 809 consensus_queue.cc:237] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } }
I20260810 18:48:29.685623 423 raft_consensus.cc:2454] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0389d4eae3394450a07764ac36867a91 in term 1.
I20260810 18:48:29.687295 844 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: TxnStatusTablet state changed. Reason: New leader 0389d4eae3394450a07764ac36867a91. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.687834 844 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:29.689038 864 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:48:29.689396 864 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:48:29.690549 32369 catalog_manager.cc:5495] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0389d4eae3394450a07764ac36867a91 (127.29.174.193). New cstate: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:29.727371 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.716194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35739 (local address 127.0.0.1:52122)
0810 18:48:29.716756 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.716779 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.716927 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.719015 (+ 2088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.719020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.719034 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:29.719345 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.719360 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.721572 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.721577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.723849 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.723868 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.724851 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.724859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.725098 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.725183 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.726499 (+ 1316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.726524 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.726652 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.726878 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352}
I20260810 18:48:29.727407 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.717325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.29.174.193:35739)
0810 18:48:29.718497 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.718503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.718529 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:48:29.718635 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.718640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.718777 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.718944 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:29.719560 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.721369 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.724016 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.724665 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.724799 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.725479 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.726335 (+ 856us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.726512 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.726701 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.726793 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":608,"threads_started":1}
I20260810 18:48:29.733942 663 raft_consensus.cc:1261] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Refusing update from remote peer 0389d4eae3394450a07764ac36867a91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.734216 423 raft_consensus.cc:1261] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Refusing update from remote peer 0389d4eae3394450a07764ac36867a91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.735042 809 consensus_queue.cc:1035] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:29.735913 809 consensus_queue.cc:1035] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:29.746757 808 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: TxnStatusTablet state changed. Reason: New leader 0389d4eae3394450a07764ac36867a91. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.747320 808 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.748106 791 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: TxnStatusTablet state changed. Reason: New leader 0389d4eae3394450a07764ac36867a91. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.748591 791 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.752460 809 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.752882 809 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:29.754627 809 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.754886 809 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 0389d4eae3394450a07764ac36867a91: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:29.756233 807 mvcc.cc:205] Tried to move back new op lower bound from 7317044059060224000 to 7317044058873180160. Current Snapshot: MvccSnapshot[applied={T|T < 7317044059060224000}]
I20260810 18:48:29.756645 791 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.757071 791 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P 1b4b7e3649fb469f9c686e727e9c71b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.759343 808 tablet_replica.cc:439] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0389d4eae3394450a07764ac36867a91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0389d4eae3394450a07764ac36867a91" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35739 } } peers { permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 } } peers { permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 } } }
I20260810 18:48:29.759675 808 tablet_replica.cc:442] T 592a6a6141da438e84418f375739ddf9 P e783debcbbd34fd98af6dcf8294c47fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:29.778191 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.769378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.29.174.193:35739)
0810 18:48:29.769556 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.769562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.769999 (+ 437us) server_negotiation.cc:407] Connection header received
0810 18:48:29.770220 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.770230 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.770321 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.770460 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:29.771348 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.772845 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.775496 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.776459 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.776603 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.777266 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.777406 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.777515 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.777677 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.777761 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:29.805174 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.793473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:43465 (local address 127.29.174.193:35739)
0810 18:48:29.793732 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.793738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.799175 (+ 5437us) server_negotiation.cc:407] Connection header received
0810 18:48:29.799458 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.799462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.799519 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.799666 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:29.800527 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.801375 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.803188 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.803737 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.803844 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.804303 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.804402 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.804479 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.804599 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.804665 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 18:48:29.805408 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.793891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35739 (local address 127.29.174.193:43465)
0810 18:48:29.799062 (+ 5171us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:29.799084 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:29.799284 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:29.799624 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:29.799629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:29.799640 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:29.800386 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.800397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.801491 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.801497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.803050 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:29.803064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.803916 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.803921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:29.804062 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.804115 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:29.804497 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:29.804508 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:48:29.804574 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 18:48:29.804725 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4951,"thread_start_us":4724,"threads_started":1}
I20260810 18:48:29.824159 663 raft_consensus.cc:1261] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Refusing update from remote peer 0389d4eae3394450a07764ac36867a91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.825093 423 raft_consensus.cc:1261] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Refusing update from remote peer 0389d4eae3394450a07764ac36867a91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:29.825090 844 consensus_queue.cc:1035] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e783debcbbd34fd98af6dcf8294c47fb" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:29.825850 877 consensus_queue.cc:1035] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4b7e3649fb469f9c686e727e9c71b1" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:29.840665 813 mvcc.cc:205] Tried to move back new op lower bound from 7317044059432116224 to 7317044058185097216. Current Snapshot: MvccSnapshot[applied={T|T < 7317044059432116224}]
I20260810 18:48:29.917790 32371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:29.918085 32371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:29.921029 32371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3629473b56984372a60edcd0c8726938: Sending DeleteTablet for 3 replicas of tablet d1eaf6a13d2446268a35c983f2491bb6
I20260810 18:48:29.922314 378 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:39400
I20260810 18:48:29.922358 620 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:43250
I20260810 18:48:29.922904 32601 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eaf6a13d2446268a35c983f2491bb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:52110
I20260810 18:48:29.943842 883 tablet_replica.cc:330] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: stopping tablet replica
I20260810 18:48:29.944075 883 raft_consensus.cc:2229] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:29.944492 883 raft_consensus.cc:2258] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.945761 885 tablet_replica.cc:330] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: stopping tablet replica
I20260810 18:48:29.945812 883 ts_tablet_manager.cc:1905] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.946769 885 raft_consensus.cc:2229] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:29.947742 885 raft_consensus.cc:2258] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.948980 885 ts_tablet_manager.cc:1905] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.957969 884 tablet_replica.cc:330] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: stopping tablet replica
I20260810 18:48:29.958312 884 raft_consensus.cc:2229] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:29.958722 884 raft_consensus.cc:2258] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:29.963568 885 ts_tablet_manager.cc:1918] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:29.964816 884 ts_tablet_manager.cc:1905] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:29.967494 883 ts_tablet_manager.cc:1918] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:29.967720 883 log.cc:1196] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/d1eaf6a13d2446268a35c983f2491bb6
I20260810 18:48:29.968209 883 ts_tablet_manager.cc:1939] T d1eaf6a13d2446268a35c983f2491bb6 P 1b4b7e3649fb469f9c686e727e9c71b1: Deleting consensus metadata
I20260810 18:48:29.969379 32342 catalog_manager.cc:4841] TS 1b4b7e3649fb469f9c686e727e9c71b1 (127.29.174.194:42659): tablet d1eaf6a13d2446268a35c983f2491bb6 (table test-table [id=e01c43f834274ab99aae65374c2238b5]) successfully deleted
Aug 10 18:48:29 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:29.973937 884 ts_tablet_manager.cc:1918] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:29.974169 885 log.cc:1196] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/d1eaf6a13d2446268a35c983f2491bb6
I20260810 18:48:29.974764 885 ts_tablet_manager.cc:1939] T d1eaf6a13d2446268a35c983f2491bb6 P 0389d4eae3394450a07764ac36867a91: Deleting consensus metadata
I20260810 18:48:29.976207 32347 catalog_manager.cc:4841] TS 0389d4eae3394450a07764ac36867a91 (127.29.174.193:35739): tablet d1eaf6a13d2446268a35c983f2491bb6 (table test-table [id=e01c43f834274ab99aae65374c2238b5]) successfully deleted
I20260810 18:48:29.976622 884 log.cc:1196] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/d1eaf6a13d2446268a35c983f2491bb6
I20260810 18:48:29.977349 884 ts_tablet_manager.cc:1939] T d1eaf6a13d2446268a35c983f2491bb6 P e783debcbbd34fd98af6dcf8294c47fb: Deleting consensus metadata
I20260810 18:48:29.978433 32347 catalog_manager.cc:4841] TS e783debcbbd34fd98af6dcf8294c47fb (127.29.174.195:37607): tablet d1eaf6a13d2446268a35c983f2491bb6 (table test-table [id=e01c43f834274ab99aae65374c2238b5]) successfully deleted
I20260810 18:48:29.992835 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:29.950312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.29.174.193:35739)
0810 18:48:29.950540 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:29.950546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:29.953483 (+ 2937us) server_negotiation.cc:407] Connection header received
0810 18:48:29.954892 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:29.954898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:29.954987 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:29.955068 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:29.958419 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.959778 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.962422 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:29.963451 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:29.963583 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:29.983265 (+ 19682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:29.983282 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:29.983301 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:29.983345 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:29.988610 (+ 5265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.989618 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.989631 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.989635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.989860 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:29.990508 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:29.990514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:29.990517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:29.991072 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:29.991406 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:29.992067 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 18:48:29.992292 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
W20260810 18:48:29.994810 32722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52128
I20260810 18:48:30.026274 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:30.007350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.29.174.254:46057)
0810 18:48:30.007589 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:30.007596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:30.010974 (+ 3378us) server_negotiation.cc:407] Connection header received
0810 18:48:30.011163 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:30.011168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:30.011236 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:30.011318 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:30.013581 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.014519 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.016001 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.016427 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.016581 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:30.019806 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:30.019812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:30.019815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:30.019850 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:30.023366 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.024130 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.024134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.024137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.024267 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.024610 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.024614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.024616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.024918 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:30.025209 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:30.025569 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 18:48:30.025843 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 18:48:30.027172 32371 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52762
I20260810 18:48:30.029412 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32498
I20260810 18:48:30.045842 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:30.031233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.29.174.193:35739)
0810 18:48:30.031444 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:30.031451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:30.032225 (+ 774us) server_negotiation.cc:407] Connection header received
0810 18:48:30.032386 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:30.032391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:30.032467 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:30.032552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:30.033695 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.034516 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.035962 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.036425 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.036530 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:30.039360 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:30.039365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:30.039368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:30.039399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:30.042775 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.043625 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.043631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.043635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.043781 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.044241 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.044246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.044248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.044506 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:30.044926 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:30.045123 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 18:48:30.045347 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 18:48:30.046485 32722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52136
W20260810 18:48:30.047051 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32498: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:30.047300 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32498
I20260810 18:48:30.079730 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32735
Aug 10 18:48:30 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:30.095031 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:30.080466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.29.174.194:42659)
0810 18:48:30.080925 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:30.080930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:30.080946 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:30.080995 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:30.080998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:30.081068 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:30.081177 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:30.082926 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.083942 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.085762 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.086252 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.086490 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:30.090208 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:30.090219 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:30.090235 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:30.090270 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:30.092780 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.093349 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.093355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.093357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.093502 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.093767 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.093770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.093772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.094074 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:30.094289 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:30.094541 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 18:48:30.094703 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
W20260810 18:48:30.095666 482 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39424
W20260810 18:48:30.096218 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32735: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:30.096379 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32735
I20260810 18:48:30.122257 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 500
Aug 10 18:48:30 dist-test-slave-6zbq krb5kdc[32297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:30.136505 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:30.123255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.29.174.195:37607)
0810 18:48:30.123687 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:30.123692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:30.123708 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:48:30.123769 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:30.123773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:30.123846 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:30.123937 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:30.125222 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.125946 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.127200 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:30.127627 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:30.127717 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:30.131506 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:30.131519 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:30.131536 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:30.131571 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:30.134148 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.134830 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.134838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.134841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.134992 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:30.135277 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:30.135280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:30.135282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:30.135604 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:30.135829 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:30.136082 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 18:48:30.136267 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":231,"threads_started":1}
W20260810 18:48:30.137096 740 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43266
W20260810 18:48:30.137863 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 500: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:30.137995 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 500
I20260810 18:48:30.160569 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32316
I20260810 18:48:30.161706 32449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:30.274482 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32316
2026-08-10T18:48:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5070 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:30 dist-test-slave-6zbq krb5kdc[903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:30 dist-test-slave-6zbq krb5kdc[903](info): set up 2 sockets
Aug 10 18:48:30 dist-test-slave-6zbq krb5kdc[903](info): commencing operation
krb5kdc: starting...
W20260810 18:48:31.496241 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:31 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:31.701638 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39425
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.016070 923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.016341 923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.016458 923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.024037 923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:32.024147 923 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:48:32.024195 923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.024253 923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.024302 923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:32.024369 923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:32.034016 923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39425
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:32.035266 923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.036530 923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.047504 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.047577 937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.047552 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.049015 923 server_base.cc:1034] running on GCE node
I20260810 18:48:32.049746 923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.051049 923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.052266 923 hybrid_clock.cc:648] HybridClock initialized: now 1786387712052206 us; error 32 us; skew 500 ppm
I20260810 18:48:32.052565 923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:32 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.055541 923 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:32.056845 923 webserver.cc:460] Webserver started at http://127.29.174.254:46883/ using document root <none> and password file <none>
I20260810 18:48:32.057471 923 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.057569 923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.057854 923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.060633 923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "7ce6726a946e42d2a6b875371a5d0714"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "0abbfa7a24584d9f4b2f6adcc084c561"
server_key_iv: "725c52a87b8e001ad8dd1eb45f82fc88"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.061373 923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "7ce6726a946e42d2a6b875371a5d0714"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "0abbfa7a24584d9f4b2f6adcc084c561"
server_key_iv: "725c52a87b8e001ad8dd1eb45f82fc88"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.066326 923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260810 18:48:32.069903 947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.071138 923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:48:32.071295 923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "7ce6726a946e42d2a6b875371a5d0714"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "0abbfa7a24584d9f4b2f6adcc084c561"
server_key_iv: "725c52a87b8e001ad8dd1eb45f82fc88"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.071489 923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.088774 923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.092618 923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.092854 923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.115326 923 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39425
I20260810 18:48:32.115306 1047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39425 every 8 connection(s)
I20260810 18:48:32.117074 923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:32.120462 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 923
I20260810 18:48:32.121098 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:32.122076 30395 external_mini_cluster.cc:1428] Setting key 2091d0500e7267b5610540f6eaaeef4b
I20260810 18:48:32.127622 1048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:32.141525 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Bootstrap starting.
Aug 10 18:48:32 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:32.145349 1048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:32.146775 1048 log.cc:824] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:32.150940 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: No bootstrap required, opened a new log
I20260810 18:48:32.152768 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.125347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.29.174.254:39425)
0810 18:48:32.126227 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.126243 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.126299 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:48:32.128626 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.128659 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.129188 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.129668 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.132608 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.134844 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.136533 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.138033 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.138326 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.143358 (+ 5032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.143371 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.143393 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.143446 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.147351 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.148528 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.148537 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.148547 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.148785 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.149249 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.149254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.149258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.150103 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.150456 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.150869 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.151359 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":384,"threads_started":1}
I20260810 18:48:32.160296 1048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } }
I20260810 18:48:32.160916 1048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:32.161046 1048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce6726a946e42d2a6b875371a5d0714, State: Initialized, Role: FOLLOWER
I20260810 18:48:32.161695 1048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } }
I20260810 18:48:32.162030 1048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:32.162186 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:32.162377 1048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:32.164620 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } }
I20260810 18:48:32.165122 1048 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce6726a946e42d2a6b875371a5d0714; no voters:
I20260810 18:48:32.165688 1048 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:32.165859 1053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:32.166952 1053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [term 1 LEADER]: Becoming Leader. State: Replica: 7ce6726a946e42d2a6b875371a5d0714, State: Running, Role: LEADER
I20260810 18:48:32.167505 1053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } }
I20260810 18:48:32.167908 1048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:32.170089 1055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce6726a946e42d2a6b875371a5d0714. Latest consensus state: current_term: 1 leader_uuid: "7ce6726a946e42d2a6b875371a5d0714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } } }
I20260810 18:48:32.170476 1055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:32.170933 1054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce6726a946e42d2a6b875371a5d0714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce6726a946e42d2a6b875371a5d0714" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39425 } } }
I20260810 18:48:32.171181 1054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:32.172477 1060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:32.177906 1060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:32.188584 1060 catalog_manager.cc:1347] Generated new cluster ID: 0af050140c8e4ef19bca1333695aa9a8
I20260810 18:48:32.188733 1060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:32.202033 1060 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:32.203514 1060 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:32.212291 1060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Generated new TSK 0
I20260810 18:48:32.213526 1060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:32.287928 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39425
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:32.666954 1083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:32.667249 1083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:32.667394 1083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:32.674883 1083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:32.674979 1083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:32.675091 1083 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:32.675158 1083 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:32.675262 1083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:32.684828 1083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:39425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:32.685974 1083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:32.687249 1083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:32.699877 1093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.699915 1092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:32.699841 1095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:32.700434 1083 server_base.cc:1034] running on GCE node
I20260810 18:48:32.700989 1083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:32.703860 1083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:32.705060 1083 hybrid_clock.cc:648] HybridClock initialized: now 1786387712705009 us; error 46 us; skew 500 ppm
I20260810 18:48:32.705346 1083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:32 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:32.708278 1083 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:32.709544 1083 webserver.cc:460] Webserver started at http://127.29.174.193:36771/ using document root <none> and password file <none>
I20260810 18:48:32.710139 1083 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:32.710237 1083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:32.710536 1083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:32.713522 1083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "bc8744abdf9a4109906540822ab6410f"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "06be696f225aa6d57c0c47239d1a737d"
server_key_iv: "32ae8a8caa43a2c7c2be79e92653d0ba"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.714231 1083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "bc8744abdf9a4109906540822ab6410f"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "06be696f225aa6d57c0c47239d1a737d"
server_key_iv: "32ae8a8caa43a2c7c2be79e92653d0ba"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.719298 1083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260810 18:48:32.723805 1103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.725206 1083 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:32.725353 1083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "bc8744abdf9a4109906540822ab6410f"
format_stamp: "Formatted at 2026-08-10 18:48:32 on dist-test-slave-6zbq"
server_key: "06be696f225aa6d57c0c47239d1a737d"
server_key_iv: "32ae8a8caa43a2c7c2be79e92653d0ba"
server_key_version: "encryptionkey@0"
I20260810 18:48:32.725529 1083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:32.744036 1083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:32.748224 1083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:32.748483 1083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:32.751108 1083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:32.751201 1083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:32.751307 1083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:32.751377 1083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:32 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:32.773726 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.751642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:44585 (local address 127.29.174.254:39425)
0810 18:48:32.752203 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.752211 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.753306 (+ 1095us) server_negotiation.cc:407] Connection header received
0810 18:48:32.755590 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.755596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.755692 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.755836 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.757830 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.758971 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.761259 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.761893 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.762034 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.766729 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.766734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.766738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.766773 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.770169 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.771089 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.771095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.771099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.771271 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.771759 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.771765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.771769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.772061 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.772220 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.773207 (+ 987us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.773376 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":312,"threads_started":1}
I20260810 18:48:32.775300 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.752277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.193:44585)
0810 18:48:32.753138 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:32.753187 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:32.755240 (+ 2053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:32.756145 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:32.756161 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:32.756858 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:32.757614 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.757645 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.759143 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.759149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.761098 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.761116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.762156 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.762165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.762458 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.763177 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:32.763211 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:32.766506 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:32.770330 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.770367 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.770385 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.770959 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.771384 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.771388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.771390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.771618 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.772211 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:32.772224 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:32.772741 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 18:48:32.773069 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 18:48:32.773663 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":356,"threads_started":1}
I20260810 18:48:32.798277 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.782673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.193:40191)
0810 18:48:32.783300 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:32.783318 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:32.783448 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:32.784032 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:32.784039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:32.784967 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:32.785283 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.785297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.787259 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.787265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.788783 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:32.788802 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.789772 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.789776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:32.789907 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.790643 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:32.790671 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:32.791631 (+ 960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:32.795759 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.795764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.795767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.796192 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.796603 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:32.796606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:32.796608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:32.796724 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:32.797382 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:32.797386 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:32.797565 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 18:48:32.797700 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 18:48:32.797950 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
I20260810 18:48:32.798493 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:32.782295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:40191 (local address 127.29.174.254:39425)
0810 18:48:32.782525 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:32.782532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:32.783403 (+ 871us) server_negotiation.cc:407] Connection header received
0810 18:48:32.783559 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:32.783565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:32.783755 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:32.783886 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:32.785591 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.787063 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.788953 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:32.789595 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:32.789733 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:32.791842 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:32.791850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:32.791856 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:32.791900 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:32.795584 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.796313 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.796320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.796324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.796498 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:32.796823 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:32.796828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:32.796832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:32.797237 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:32.797398 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:32.797821 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 18:48:32.798083 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 18:48:32.805001 1083 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:43217
I20260810 18:48:32.805051 1308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:43217 every 8 connection(s)
I20260810 18:48:32.807080 1083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:32.809273 1309 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39425
I20260810 18:48:32.809489 1309 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:32.810101 1309 heartbeater.cc:511] Master 127.29.174.254:39425 requested a full tablet report, sending...
I20260810 18:48:32.811932 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1083
I20260810 18:48:32.812142 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:32.812137 970 ts_manager.cc:194] Registered new tserver with Master: bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217)
I20260810 18:48:32.812947 30395 external_mini_cluster.cc:1428] Setting key 2c94434508708cff56266d09b7305957
I20260810 18:48:32.814065 970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:44585
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:32.858769 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39425
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:33.187484 1318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:33.187741 1318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:33.187850 1318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:33.194761 1318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:33.194864 1318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:33.194957 1318 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:33.195010 1318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:33.195077 1318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:33.204890 1318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:39425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:33.206063 1318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:33.207288 1318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:33.219477 1324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:33.219511 1330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:33.219703 1327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:33.220688 1318 server_base.cc:1034] running on GCE node
I20260810 18:48:33.221359 1318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:33.222306 1318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:33.223541 1318 hybrid_clock.cc:648] HybridClock initialized: now 1786387713223487 us; error 41 us; skew 500 ppm
I20260810 18:48:33.223835 1318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:33.230103 1318 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:33.231594 1318 webserver.cc:460] Webserver started at http://127.29.174.194:38283/ using document root <none> and password file <none>
I20260810 18:48:33.232235 1318 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:33.232360 1318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:33.232649 1318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:33.235481 1318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "714bae5339f44945bc4ef2d374563cbb"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "60eaaba86149113234a0ff816d5d8321"
server_key_iv: "f2d66255018bfe8d0c44507467ff7fc8"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.236189 1318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "714bae5339f44945bc4ef2d374563cbb"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "60eaaba86149113234a0ff816d5d8321"
server_key_iv: "f2d66255018bfe8d0c44507467ff7fc8"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.241022 1318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:33.244243 1344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:33.245404 1318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:33.245548 1318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "714bae5339f44945bc4ef2d374563cbb"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "60eaaba86149113234a0ff816d5d8321"
server_key_iv: "f2d66255018bfe8d0c44507467ff7fc8"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.245707 1318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:33.272961 1318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:33.277204 1318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:33.277477 1318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:33.280187 1318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:33.280303 1318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:33.280433 1318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:33.280514 1318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:33.300534 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.280514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:38769 (local address 127.29.174.254:39425)
0810 18:48:33.280704 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.280710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.282021 (+ 1311us) server_negotiation.cc:407] Connection header received
0810 18:48:33.283987 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.283993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.284077 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.284162 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.286101 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.287106 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.289562 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.290161 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.290280 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.294496 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.294503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.294508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.294549 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.297094 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.297897 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.297903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.297906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.298056 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.298670 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.298676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.298679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.299002 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.299293 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.299977 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.300192 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:48:33.301450 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.281038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.194:38769)
0810 18:48:33.281851 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.281896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.283722 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.284393 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.284404 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.285058 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:33.285866 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.285895 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.287258 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.287264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.289358 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.289374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.290474 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.290484 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.290677 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.291295 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:33.291321 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:33.294220 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:33.297352 (+ 3132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.297361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.297376 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.297783 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.298233 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.298238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.298242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.298509 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.299217 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:33.299249 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:33.299589 (+ 340us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.299832 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.300313 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":337,"threads_started":1}
I20260810 18:48:33.322187 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.308663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:40109 (local address 127.29.174.254:39425)
0810 18:48:33.308833 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.308839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.309657 (+ 818us) server_negotiation.cc:407] Connection header received
0810 18:48:33.309808 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.309814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.309875 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.309952 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.311071 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.311924 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.314156 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.314574 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.314664 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.316888 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.316893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.316896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.316924 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.319615 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.320286 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.320291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.320296 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.320425 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.320753 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.320758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.320761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.321142 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.321298 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.321668 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.321862 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:48:33.322307 1484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.308880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.194:40109)
0810 18:48:33.309563 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.309581 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.309699 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.309986 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.309990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.310540 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:33.310934 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.310945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.312058 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.312064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.314001 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.314018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.314761 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.314768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.314945 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.315526 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:33.315549 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:33.316694 (+ 1145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:33.319731 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.319736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.319741 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.320171 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.320526 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.320531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.320535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.320655 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.321252 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:33.321258 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:33.321445 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.321563 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.321892 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":394,"threads_started":1}
I20260810 18:48:33.333554 1318 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:43931
I20260810 18:48:33.333595 1565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:43931 every 8 connection(s)
I20260810 18:48:33.334719 1318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:33.336571 1568 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39425
I20260810 18:48:33.336758 1568 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:33.337337 1568 heartbeater.cc:511] Master 127.29.174.254:39425 requested a full tablet report, sending...
I20260810 18:48:33.338583 969 ts_manager.cc:194] Registered new tserver with Master: 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931)
I20260810 18:48:33.339466 969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:38769
I20260810 18:48:33.339860 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1318
I20260810 18:48:33.340119 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:33.341048 30395 external_mini_cluster.cc:1428] Setting key 4ac081824b633b181e8ad5ab4777a90b
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:33.391916 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39425
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:33.707878 1575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:33.708137 1575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:33.708256 1575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:33.715257 1575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:33.715351 1575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:33.715441 1575 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:48:33.715492 1575 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:48:33.715557 1575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:33.724856 1575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:39425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:33.725960 1575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:33.727181 1575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:33.739332 1582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:33.739367 1583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:33.739390 1591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:33.740032 1575 server_base.cc:1034] running on GCE node
I20260810 18:48:33.740568 1575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:33.741385 1575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:33.742573 1575 hybrid_clock.cc:648] HybridClock initialized: now 1786387713742510 us; error 46 us; skew 500 ppm
I20260810 18:48:33.742848 1575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:33.745695 1575 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:33.747045 1575 webserver.cc:460] Webserver started at http://127.29.174.195:38767/ using document root <none> and password file <none>
I20260810 18:48:33.747625 1575 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:33.747721 1575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:33.747957 1575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:33.750797 1575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "790b29ce942540f4a50a3997dd93ff50"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "0311d4cc02b2511de2a072000831406d"
server_key_iv: "2b9017d8b25d87bbcf34328a5cbc12cd"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.751482 1575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "790b29ce942540f4a50a3997dd93ff50"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "0311d4cc02b2511de2a072000831406d"
server_key_iv: "2b9017d8b25d87bbcf34328a5cbc12cd"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.756162 1575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:33.759253 1603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:33.760412 1575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:33.760557 1575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "790b29ce942540f4a50a3997dd93ff50"
format_stamp: "Formatted at 2026-08-10 18:48:33 on dist-test-slave-6zbq"
server_key: "0311d4cc02b2511de2a072000831406d"
server_key_iv: "2b9017d8b25d87bbcf34328a5cbc12cd"
server_key_version: "encryptionkey@0"
I20260810 18:48:33.760722 1575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:33.780493 1575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:33.783989 1575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:33.784245 1575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:33.786723 1575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:33.786839 1575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:33.786960 1575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:33.787042 1575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:33.809830 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.787068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:32829 (local address 127.29.174.254:39425)
0810 18:48:33.787230 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.787235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.788598 (+ 1363us) server_negotiation.cc:407] Connection header received
0810 18:48:33.790520 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.790525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.790606 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.790689 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.792693 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.793664 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.796015 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.796536 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.796653 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.800952 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.800959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.800964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.801000 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.804071 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.805050 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.805055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.805059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.805224 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.805857 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.805862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.805865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.806185 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.806323 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.809285 (+ 2962us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.809506 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:33.809937 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.787682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.195:32829)
0810 18:48:33.788461 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.788494 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.790239 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.790960 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.790974 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.791696 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:33.792491 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.792512 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.793819 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.793829 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.795883 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.795898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.796763 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.796772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.796962 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.797694 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:33.797730 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:33.800746 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:33.804238 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.804247 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.804267 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.804902 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.805373 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.805379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.805383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.805689 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.806429 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:33.806438 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:33.806885 (+ 447us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.807203 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.807747 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":326,"threads_started":1}
I20260810 18:48:33.817519 1309 heartbeater.cc:503] Master 127.29.174.254:39425 was elected leader, sending a full tablet report...
I20260810 18:48:33.826972 1575 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:45111
I20260810 18:48:33.827280 1808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:45111 every 8 connection(s)
I20260810 18:48:33.828459 1575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:33.830739 1809 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39425
I20260810 18:48:33.830950 1809 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:33.831502 1757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.816071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.29.174.195:39993)
0810 18:48:33.816761 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.816776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.816885 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.817228 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.817233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.817859 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:33.818587 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.818603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.819979 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.819988 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.821860 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.821878 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.822947 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.822953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.823139 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.823748 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:33.823771 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:33.825006 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:33.828806 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.828810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.828813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.829377 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.829784 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:33.829789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:33.829793 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:33.829934 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:33.830558 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:33.830563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:33.830745 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.830894 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.831158 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":424,"threads_started":1}
I20260810 18:48:33.831591 1809 heartbeater.cc:511] Master 127.29.174.254:39425 requested a full tablet report, sending...
I20260810 18:48:33.832201 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.815927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:39993 (local address 127.29.174.254:39425)
0810 18:48:33.816081 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.816087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.816864 (+ 777us) server_negotiation.cc:407] Connection header received
0810 18:48:33.816989 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.816995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.817080 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.817179 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.818770 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.819804 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.822063 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.822746 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.822877 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.825277 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.825283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.825288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.825323 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.828681 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.829502 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.829508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.829512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.829673 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.830075 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.830080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.830083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.830438 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.830578 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.831773 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.831972 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:48:33.832921 970 ts_manager.cc:194] Registered new tserver with Master: 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:33.833614 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1575
I20260810 18:48:33.833796 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:33.833858 970 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:32829
I20260810 18:48:33.834316 30395 external_mini_cluster.cc:1428] Setting key 293bfee628987b37c88a582a221b6a47
I20260810 18:48:33.837106 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:33 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:33.886152 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.873170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.29.174.254:39425)
0810 18:48:33.873372 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.873377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.873548 (+ 171us) server_negotiation.cc:407] Connection header received
0810 18:48:33.873744 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.873748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.873813 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.873914 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:33.875325 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.876033 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.877296 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.877698 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.877804 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.881463 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:33.881468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:33.881471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:33.881502 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:33.883777 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.884363 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.884367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.884369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.884489 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:33.884846 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:33.884851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:33.884853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:33.885102 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:33.885336 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.885608 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.885822 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:33.889870 970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:33.896420 970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:33.920214 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.908440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.29.174.193:43217)
0810 18:48:33.909323 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.909331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.910219 (+ 888us) server_negotiation.cc:407] Connection header received
0810 18:48:33.910389 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.910399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.910693 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.910907 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:33.911924 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.913556 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.917245 (+ 3689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.918722 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.918895 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.919073 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.919673 (+ 600us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.919847 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":597,"threads_started":1}
I20260810 18:48:33.920349 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.907943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:55336)
0810 18:48:33.909841 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.909883 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.910061 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.911192 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.911201 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.911237 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:33.911680 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.911694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.913740 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.913746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.916243 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.916260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.919196 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.919203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.919430 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.919486 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.919618 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.919833 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1667,"thread_start_us":1538,"threads_started":1}
I20260810 18:48:33.924163 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.907500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:36812)
0810 18:48:33.908306 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.908346 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.908522 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.917459 (+ 8937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.917468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.917495 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:33.918041 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.918052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.919506 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.919512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.921575 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.921595 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.923302 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.923309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.923539 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.923576 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.923672 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.923798 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":33,"thread_start_us":398,"threads_started":1}
I20260810 18:48:33.924192 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.907637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.29.174.194:43931)
0810 18:48:33.916842 (+ 9205us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.916854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.916883 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:48:33.916978 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.916985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.917242 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.917854 (+ 612us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:33.918232 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.919380 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.921783 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.923081 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.923247 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.923399 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.923743 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.923904 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9006,"thread_start_us":8863,"threads_started":1}
I20260810 18:48:33.924499 1181 tablet_service.cc:1467] Processing CreateTablet for tablet 752d0eee1f2e485ea7a948460555bccf (DEFAULT_TABLE table=test-table [id=e0f7606befc844d19cdba7f14cc3270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:33.925908 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752d0eee1f2e485ea7a948460555bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:33.926654 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.908662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:45111 (local address 127.0.0.1:47236)
0810 18:48:33.917816 (+ 9154us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:33.917835 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:33.918049 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:33.918960 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:33.918965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:33.918989 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:33.919396 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.919408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.921946 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.921951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.923585 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:33.923597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.925886 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.925893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:33.926032 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.926059 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:48:33.926142 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:48:33.926259 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9019,"mutex_wait_us":21,"thread_start_us":8840,"threads_started":1}
I20260810 18:48:33.926688 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:33.909134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.29.174.195:45111)
0810 18:48:33.917326 (+ 8192us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:33.917335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:33.917935 (+ 600us) server_negotiation.cc:407] Connection header received
0810 18:48:33.918229 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:33.918237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:33.918477 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:33.918649 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:33.919623 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.921783 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.923734 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:33.925100 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:33.925266 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:33.925424 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:33.926189 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 18:48:33.926391 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7995,"thread_start_us":7776,"threads_started":1}
I20260810 18:48:33.927582 1444 tablet_service.cc:1467] Processing CreateTablet for tablet 752d0eee1f2e485ea7a948460555bccf (DEFAULT_TABLE table=test-table [id=e0f7606befc844d19cdba7f14cc3270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:33.928884 1444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752d0eee1f2e485ea7a948460555bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:33.929612 1688 tablet_service.cc:1467] Processing CreateTablet for tablet 752d0eee1f2e485ea7a948460555bccf (DEFAULT_TABLE table=test-table [id=e0f7606befc844d19cdba7f14cc3270f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:33.930860 1688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752d0eee1f2e485ea7a948460555bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:33.940387 1845 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Bootstrap starting.
I20260810 18:48:33.942305 1847 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Bootstrap starting.
I20260810 18:48:33.944296 1845 tablet_bootstrap.cc:654] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:33.945240 1845 log.cc:824] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:33.945698 1847 tablet_bootstrap.cc:654] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:33.946907 1847 log.cc:824] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:33.948323 1848 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Bootstrap starting.
I20260810 18:48:33.952162 1847 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: No bootstrap required, opened a new log
I20260810 18:48:33.952205 1845 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: No bootstrap required, opened a new log
I20260810 18:48:33.952402 1848 tablet_bootstrap.cc:654] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:33.952430 1847 ts_tablet_manager.cc:1397] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:48:33.952432 1845 ts_tablet_manager.cc:1397] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260810 18:48:33.953578 1848 log.cc:824] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:33.956629 1848 tablet_bootstrap.cc:492] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: No bootstrap required, opened a new log
I20260810 18:48:33.956871 1848 ts_tablet_manager.cc:1397] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 18:48:33.959878 1845 raft_consensus.cc:348] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.960219 1845 raft_consensus.cc:374] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:33.960310 1845 raft_consensus.cc:729] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Initialized, Role: FOLLOWER
I20260810 18:48:33.960844 1845 consensus_queue.cc:260] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.962544 1845 ts_tablet_manager.cc:1428] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 18:48:33.962364 1847 raft_consensus.cc:348] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.962756 1847 raft_consensus.cc:374] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:33.962847 1847 raft_consensus.cc:729] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Initialized, Role: FOLLOWER
I20260810 18:48:33.963522 1847 consensus_queue.cc:260] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.964615 1809 heartbeater.cc:503] Master 127.29.174.254:39425 was elected leader, sending a full tablet report...
I20260810 18:48:33.965224 1847 ts_tablet_manager.cc:1428] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 18:48:33.966728 1848 raft_consensus.cc:348] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.967129 1848 raft_consensus.cc:374] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:33.967228 1848 raft_consensus.cc:729] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:33.967772 1848 consensus_queue.cc:260] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:33.968811 1568 heartbeater.cc:503] Master 127.29.174.254:39425 was elected leader, sending a full tablet report...
I20260810 18:48:33.969307 1848 ts_tablet_manager.cc:1428] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
W20260810 18:48:34.059460 1311 tablet.cc:2369] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:34.064512 1853 raft_consensus.cc:482] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:34.064736 1853 raft_consensus.cc:504] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:34.066372 1853 leader_election.cc:288] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.073048 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.066582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.29.174.193:39943)
0810 18:48:34.067112 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.067130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.067288 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.067907 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.067912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.067943 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.068372 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.068388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.069524 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.069530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.071152 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.071165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.072319 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.072326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.072494 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.072531 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.072624 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.072745 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":274,"threads_started":1}
I20260810 18:48:34.073065 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.066666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:39943 (local address 127.29.174.194:43931)
0810 18:48:34.066889 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.066895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.067238 (+ 343us) server_negotiation.cc:407] Connection header received
0810 18:48:34.067474 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.067480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.067608 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.067677 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.068522 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.069406 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.071303 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.072104 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.072247 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.072302 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.072653 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.072793 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:34.073388 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.067481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:36577 (local address 127.29.174.195:45111)
0810 18:48:34.067670 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.067676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.067862 (+ 186us) server_negotiation.cc:407] Connection header received
0810 18:48:34.068021 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.068026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.068101 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.068193 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.068810 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.069634 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.071452 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.072445 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.072595 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.072660 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.073027 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.073156 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:34.073484 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.067206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:45111 (local address 127.29.174.193:36577)
0810 18:48:34.067765 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.067779 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.067902 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.068216 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.068219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.068243 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.068649 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.068662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.069780 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.069783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.071305 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.071320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.072660 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.072666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.072856 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.072895 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.072989 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.073110 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":263,"threads_started":1}
I20260810 18:48:34.074383 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752d0eee1f2e485ea7a948460555bccf" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50" is_pre_election: true
I20260810 18:48:34.074378 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752d0eee1f2e485ea7a948460555bccf" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb" is_pre_election: true
I20260810 18:48:34.074858 1721 raft_consensus.cc:2454] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 0.
I20260810 18:48:34.074877 1486 raft_consensus.cc:2454] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 0.
I20260810 18:48:34.075553 1111 leader_election.cc:302] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:34.075907 1853 raft_consensus.cc:2781] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:34.076045 1853 raft_consensus.cc:482] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:34.076139 1853 raft_consensus.cc:3037] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.078540 1853 raft_consensus.cc:504] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
I20260810 18:48:34.079118 1853 leader_election.cc:288] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 election: Requested vote from peers 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.079514 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752d0eee1f2e485ea7a948460555bccf" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb"
I20260810 18:48:34.079574 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752d0eee1f2e485ea7a948460555bccf" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50"
I20260810 18:48:34.079687 1486 raft_consensus.cc:3037] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.079756 1721 raft_consensus.cc:3037] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:48:34.080394 1812 tablet.cc:2369] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:34.082020 1721 raft_consensus.cc:2454] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 1.
I20260810 18:48:34.082237 1486 raft_consensus.cc:2454] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 1.
I20260810 18:48:34.082396 1111 leader_election.cc:302] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 790b29ce942540f4a50a3997dd93ff50, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:34.082700 1853 raft_consensus.cc:2781] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:34.083096 1853 raft_consensus.cc:686] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 1 LEADER]: Becoming Leader. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Running, Role: LEADER
I20260810 18:48:34.083585 1853 consensus_queue.cc:237] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } }
W20260810 18:48:34.086485 1569 tablet.cc:2369] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:34.087641 969 catalog_manager.cc:5495] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc8744abdf9a4109906540822ab6410f (127.29.174.193). New cstate: current_term: 1 leader_uuid: "bc8744abdf9a4109906540822ab6410f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } health_report { overall_health: HEALTHY } } }
Aug 10 18:48:34 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387712, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:34.161437 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.148828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39425 (local address 127.0.0.1:33388)
0810 18:48:34.149429 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.149445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.149599 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.149927 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.149930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.150633 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:34.150909 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.150918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.151878 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.151882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.152850 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.152861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.153607 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.153612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.153748 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.154371 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:34.154397 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:34.156458 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:34.159028 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:34.159039 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:34.159045 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:34.159442 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:34.159751 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:34.159754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:34.159757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:34.159895 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:34.160521 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:34.160528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:34.160706 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.160853 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.161055 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":317,"threads_started":1}
I20260810 18:48:34.161481 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.148782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.29.174.254:39425)
0810 18:48:34.148964 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.148974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.149524 (+ 550us) server_negotiation.cc:407] Connection header received
0810 18:48:34.149720 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.149724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.149806 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.149881 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:34.151038 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.151761 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.152969 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.153387 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.153475 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.156627 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:34.156632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:34.156635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:34.156663 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:34.158904 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.159535 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.159539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.159542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.159664 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:34.159978 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:34.159982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:34.159984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:34.160379 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:34.160501 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.160935 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.161182 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":55}
I20260810 18:48:34.166803 969 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:33388:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:48:34.167865 969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:34.174031 1181 tablet_service.cc:1467] Processing CreateTablet for tablet c21f62f7003c4e8e9c33f3c1ec55baf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75872474edd04d8da824e7fce828e0c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:34.174367 1444 tablet_service.cc:1467] Processing CreateTablet for tablet c21f62f7003c4e8e9c33f3c1ec55baf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75872474edd04d8da824e7fce828e0c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:34.174464 1688 tablet_service.cc:1467] Processing CreateTablet for tablet c21f62f7003c4e8e9c33f3c1ec55baf2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=75872474edd04d8da824e7fce828e0c4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:48:34.175060 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21f62f7003c4e8e9c33f3c1ec55baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.175186 1444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21f62f7003c4e8e9c33f3c1ec55baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.175302 1688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c21f62f7003c4e8e9c33f3c1ec55baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.181301 1848 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: Bootstrap starting.
I20260810 18:48:34.182324 1847 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: Bootstrap starting.
I20260810 18:48:34.182938 1848 tablet_bootstrap.cc:654] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.184012 1847 tablet_bootstrap.cc:654] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.184959 1845 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: Bootstrap starting.
I20260810 18:48:34.185091 1848 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: No bootstrap required, opened a new log
I20260810 18:48:34.185221 1848 ts_tablet_manager.cc:1397] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 18:48:34.185963 1848 raft_consensus.cc:348] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.186197 1848 raft_consensus.cc:374] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.186311 1848 raft_consensus.cc:729] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.186605 1845 tablet_bootstrap.cc:654] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.186575 1848 consensus_queue.cc:260] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.187019 1856 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.187361 1856 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.187628 1847 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: No bootstrap required, opened a new log
I20260810 18:48:34.187769 1847 ts_tablet_manager.cc:1397] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260810 18:48:34.188359 1848 ts_tablet_manager.cc:1428] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:34.188508 1847 raft_consensus.cc:348] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.188750 1847 raft_consensus.cc:374] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.188841 1847 raft_consensus.cc:729] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.189062 1847 consensus_queue.cc:260] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.189668 1847 ts_tablet_manager.cc:1428] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 18:48:34.189553 1854 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.189919 1854 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.190241 1845 tablet_bootstrap.cc:492] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: No bootstrap required, opened a new log
I20260810 18:48:34.190403 1845 ts_tablet_manager.cc:1397] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:48:34.190740 1856 raft_consensus.cc:482] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:34.190912 1856 raft_consensus.cc:504] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.191176 1845 raft_consensus.cc:348] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.191931 1845 raft_consensus.cc:374] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.192025 1845 raft_consensus.cc:729] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.192237 1845 consensus_queue.cc:260] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.192694 1856 leader_election.cc:288] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.192625 1853 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.192888 1853 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.193789 1845 ts_tablet_manager.cc:1428] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 18:48:34.201927 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.194701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:35217 (local address 127.29.174.193:43217)
0810 18:48:34.194895 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.194902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.194923 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:48:34.195073 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.195079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.195145 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.195227 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.195927 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.196952 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.199457 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.200638 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.200804 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.200876 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.201538 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.201676 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:48:34.201946 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.193389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:56299 (local address 127.29.174.195:45111)
0810 18:48:34.193544 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.193549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.194494 (+ 945us) server_negotiation.cc:407] Connection header received
0810 18:48:34.194995 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.195002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.195071 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.195138 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.196028 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.196770 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.199457 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.200681 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.200819 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.200883 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.201538 (+ 655us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.201690 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:48:34.202096 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.192960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:45111 (local address 127.29.174.194:56299)
0810 18:48:34.194392 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.194410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.194614 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.195349 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.195354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.195395 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.195850 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.195863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.196903 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.196909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.199288 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.199304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.200941 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.200947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.201279 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.201371 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.201472 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.201641 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1279,"mutex_wait_us":124,"thread_start_us":1116,"threads_started":1}
I20260810 18:48:34.202467 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.194173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.29.174.194:35217)
0810 18:48:34.194842 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.194857 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.194946 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.195250 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.195255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.195281 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.195712 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.195731 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.197081 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.197085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.199288 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.199304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.200934 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.200941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.201279 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.201371 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.201472 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.201641 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":133,"thread_start_us":1255,"threads_started":1}
I20260810 18:48:34.202667 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21f62f7003c4e8e9c33f3c1ec55baf2" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50" is_pre_election: true
I20260810 18:48:34.202904 1721 raft_consensus.cc:2454] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 0.
I20260810 18:48:34.203331 1349 leader_election.cc:302] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, 790b29ce942540f4a50a3997dd93ff50; no voters:
I20260810 18:48:34.203364 1221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21f62f7003c4e8e9c33f3c1ec55baf2" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f" is_pre_election: true
I20260810 18:48:34.203780 1856 raft_consensus.cc:2781] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:34.203780 1221 raft_consensus.cc:2454] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 0.
I20260810 18:48:34.203908 1856 raft_consensus.cc:482] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:34.204002 1856 raft_consensus.cc:3037] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.206532 1856 raft_consensus.cc:504] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.207067 1856 leader_election.cc:288] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 election: Requested vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.207492 1221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21f62f7003c4e8e9c33f3c1ec55baf2" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f"
I20260810 18:48:34.207489 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c21f62f7003c4e8e9c33f3c1ec55baf2" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50"
I20260810 18:48:34.207641 1221 raft_consensus.cc:3037] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.207675 1721 raft_consensus.cc:3037] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.209553 1721 raft_consensus.cc:2454] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 1.
I20260810 18:48:34.209638 1221 raft_consensus.cc:2454] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 1.
I20260810 18:48:34.209947 1349 leader_election.cc:302] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, 790b29ce942540f4a50a3997dd93ff50; no voters:
I20260810 18:48:34.210244 1856 raft_consensus.cc:2781] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:34.210691 1856 raft_consensus.cc:686] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [term 1 LEADER]: Becoming Leader. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Running, Role: LEADER
I20260810 18:48:34.211123 1856 consensus_queue.cc:237] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.213902 1882 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: TxnStatusTablet state changed. Reason: New leader 714bae5339f44945bc4ef2d374563cbb. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.214366 1882 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:34.215091 1886 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:48:34.215353 1886 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:48:34.215677 970 catalog_manager.cc:5495] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 714bae5339f44945bc4ef2d374563cbb (127.29.174.194). New cstate: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:34.238492 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.232298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:36824)
0810 18:48:34.232504 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.232524 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.232664 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.233140 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.233143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.233153 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:34.233333 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.233345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.234866 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.234871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.235858 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.235870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.236839 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.236844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.236993 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.237047 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.238045 (+ 998us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.238056 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.238102 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.238219 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 18:48:34.238638 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.232449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36824 (local address 127.29.174.194:43931)
0810 18:48:34.232637 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.232643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.232665 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:48:34.232863 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.232869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.232967 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.233138 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:34.233471 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.234710 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.236015 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.236672 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.236817 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.237244 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.237944 (+ 700us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.238111 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.238161 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.238248 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:48:34.245863 1221 raft_consensus.cc:1261] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Refusing update from remote peer 714bae5339f44945bc4ef2d374563cbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.246127 1721 raft_consensus.cc:1261] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Refusing update from remote peer 714bae5339f44945bc4ef2d374563cbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.246793 1882 consensus_queue.cc:1035] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:34.247125 1856 consensus_queue.cc:1035] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:34.258124 1854 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: TxnStatusTablet state changed. Reason: New leader 714bae5339f44945bc4ef2d374563cbb. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.258445 1854 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.259367 1853 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: TxnStatusTablet state changed. Reason: New leader 714bae5339f44945bc4ef2d374563cbb. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.259687 1853 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.261765 1890 mvcc.cc:205] Tried to move back new op lower bound from 7317044077541765120 to 7317044077412401152. Current Snapshot: MvccSnapshot[applied={T|T < 7317044077541765120}]
I20260810 18:48:34.263681 1882 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.263952 1882 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:34.264281 1891 mvcc.cc:205] Tried to move back new op lower bound from 7317044077541765120 to 7317044077412401152. Current Snapshot: MvccSnapshot[applied={T|T < 7317044077541765120}]
I20260810 18:48:34.266314 1882 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.266578 1882 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 714bae5339f44945bc4ef2d374563cbb: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:48:34.269133 1853 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.269399 1853 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P bc8744abdf9a4109906540822ab6410f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.272336 1854 tablet_replica.cc:439] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } } }
I20260810 18:48:34.272580 1854 tablet_replica.cc:442] T c21f62f7003c4e8e9c33f3c1ec55baf2 P 790b29ce942540f4a50a3997dd93ff50: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:48:34.281275 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.275392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.29.174.193:43217)
0810 18:48:34.275567 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.275573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.275788 (+ 215us) server_negotiation.cc:407] Connection header received
0810 18:48:34.275980 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.275984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.276050 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.276180 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:34.276561 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.277383 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.279283 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.279672 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.279764 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.280259 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.280700 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.280810 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.280918 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.280980 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:48:34.296772 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.290984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:53405 (local address 127.29.174.194:43931)
0810 18:48:34.291164 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.291170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.291409 (+ 239us) server_negotiation.cc:407] Connection header received
0810 18:48:34.291611 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.291615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.291679 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.291762 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:34.292387 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.293189 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.295188 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.295611 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.295703 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.296221 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.296313 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.296378 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.296498 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.296555 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:34.297034 1907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.290706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.29.174.193:53405)
0810 18:48:34.291310 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.291330 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.291477 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.291794 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.291799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.291812 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:34.292182 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.292195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.293320 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.293326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.295052 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.295068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.295777 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.295784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.295953 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.296022 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.296400 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.296411 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.296487 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.296622 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":263,"threads_started":1}
I20260810 18:48:34.307286 1721 raft_consensus.cc:1261] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Refusing update from remote peer bc8744abdf9a4109906540822ab6410f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.307637 1486 raft_consensus.cc:1261] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Refusing update from remote peer bc8744abdf9a4109906540822ab6410f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.307999 1853 consensus_queue.cc:1035] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [LEADER]: Connected to new peer: Peer: permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:34.308279 1860 consensus_queue.cc:1035] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [LEADER]: Connected to new peer: Peer: permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:34.316424 1893 mvcc.cc:205] Tried to move back new op lower bound from 7317044077796679680 to 7317044076889878528. Current Snapshot: MvccSnapshot[applied={T|T < 7317044077796679680}]
I20260810 18:48:34.350318 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.343863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:57411 (local address 127.29.174.193:43217)
0810 18:48:34.344036 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.344041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.344776 (+ 735us) server_negotiation.cc:407] Connection header received
0810 18:48:34.344975 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.344979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.345042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.345121 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:34.345803 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.346558 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.348248 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.348829 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.348927 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.349567 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.349699 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.349760 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.349944 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.350022 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:48:34.350692 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.343935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.29.174.194:57411)
0810 18:48:34.344685 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.344705 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.344845 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.345277 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.345282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.345296 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:34.345636 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.345645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.346684 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.346690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.348068 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.348083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.349122 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.349129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.349299 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.349371 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:34.349825 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:34.349832 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.349899 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.350036 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":384,"threads_started":1}
I20260810 18:48:34.394577 970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:34.394865 970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:34.398922 970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Sending DeleteTablet for 3 replicas of tablet 752d0eee1f2e485ea7a948460555bccf
I20260810 18:48:34.400383 1444 tablet_service.cc:1514] Processing DeleteTablet for tablet 752d0eee1f2e485ea7a948460555bccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:34 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36812
I20260810 18:48:34.400384 1688 tablet_service.cc:1514] Processing DeleteTablet for tablet 752d0eee1f2e485ea7a948460555bccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:34 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:47236
I20260810 18:48:34.401561 1181 tablet_service.cc:1514] Processing DeleteTablet for tablet 752d0eee1f2e485ea7a948460555bccf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:34 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:55336
I20260810 18:48:34.408710 970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:34.410948 970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:34.421912 1178 tablet_service.cc:1467] Processing CreateTablet for tablet 35428583799b4292844efd8fed55d003 (DEFAULT_TABLE table=test-table [id=d783e53206a64885bc3a9ac12493bc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:34.422513 1178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35428583799b4292844efd8fed55d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.423758 1440 tablet_service.cc:1467] Processing CreateTablet for tablet 35428583799b4292844efd8fed55d003 (DEFAULT_TABLE table=test-table [id=d783e53206a64885bc3a9ac12493bc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:34.423964 1687 tablet_service.cc:1467] Processing CreateTablet for tablet 35428583799b4292844efd8fed55d003 (DEFAULT_TABLE table=test-table [id=d783e53206a64885bc3a9ac12493bc7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:34.424408 1687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35428583799b4292844efd8fed55d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.424418 1440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35428583799b4292844efd8fed55d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:34.431087 1847 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Bootstrap starting.
I20260810 18:48:34.432883 1847 tablet_bootstrap.cc:654] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.437160 1847 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: No bootstrap required, opened a new log
I20260810 18:48:34.437320 1847 ts_tablet_manager.cc:1397] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:48:34.438077 1845 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Bootstrap starting.
I20260810 18:48:34.438055 1847 raft_consensus.cc:348] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.438305 1847 raft_consensus.cc:374] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.438421 1847 raft_consensus.cc:729] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.438655 1847 consensus_queue.cc:260] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.439203 1847 ts_tablet_manager.cc:1428] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:34.439669 1845 tablet_bootstrap.cc:654] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.440094 1927 tablet_replica.cc:330] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: stopping tablet replica
I20260810 18:48:34.440615 1927 raft_consensus.cc:2229] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:34.441036 1927 raft_consensus.cc:2258] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:34.443130 1927 ts_tablet_manager.cc:1905] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:34.443915 1848 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Bootstrap starting.
I20260810 18:48:34.445770 1848 tablet_bootstrap.cc:654] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:34.449056 1925 tablet_replica.cc:330] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: stopping tablet replica
I20260810 18:48:34.449468 1925 raft_consensus.cc:2229] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:34.449774 1925 raft_consensus.cc:2258] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:34.451792 1848 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: No bootstrap required, opened a new log
I20260810 18:48:34.451953 1848 ts_tablet_manager.cc:1397] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 18:48:34.452595 1848 raft_consensus.cc:348] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.452831 1848 raft_consensus.cc:374] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.452929 1848 raft_consensus.cc:729] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.453171 1848 consensus_queue.cc:260] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.453830 1848 ts_tablet_manager.cc:1428] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:34.454597 1927 ts_tablet_manager.cc:1918] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:34.455662 1927 log.cc:1196] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/752d0eee1f2e485ea7a948460555bccf
I20260810 18:48:34.457598 1925 ts_tablet_manager.cc:1905] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:34.458454 1927 ts_tablet_manager.cc:1939] T 752d0eee1f2e485ea7a948460555bccf P 714bae5339f44945bc4ef2d374563cbb: Deleting consensus metadata
I20260810 18:48:34.438082 1928 tablet_replica.cc:330] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: stopping tablet replica
I20260810 18:48:34.460281 951 catalog_manager.cc:4841] TS 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931): tablet 752d0eee1f2e485ea7a948460555bccf (table test-table [id=e0f7606befc844d19cdba7f14cc3270f]) successfully deleted
I20260810 18:48:34.460582 1928 raft_consensus.cc:2229] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:34.461464 1928 raft_consensus.cc:2258] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:34.464619 1925 ts_tablet_manager.cc:1918] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:34.464787 1925 log.cc:1196] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/752d0eee1f2e485ea7a948460555bccf
I20260810 18:48:34.465268 1925 ts_tablet_manager.cc:1939] T 752d0eee1f2e485ea7a948460555bccf P 790b29ce942540f4a50a3997dd93ff50: Deleting consensus metadata
I20260810 18:48:34.465886 1845 tablet_bootstrap.cc:492] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: No bootstrap required, opened a new log
I20260810 18:48:34.466099 1845 ts_tablet_manager.cc:1397] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.000s
I20260810 18:48:34.466939 951 catalog_manager.cc:4841] TS 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111): tablet 752d0eee1f2e485ea7a948460555bccf (table test-table [id=e0f7606befc844d19cdba7f14cc3270f]) successfully deleted
I20260810 18:48:34.466949 1845 raft_consensus.cc:348] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.467346 1845 raft_consensus.cc:374] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:34.467484 1845 raft_consensus.cc:729] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Initialized, Role: FOLLOWER
I20260810 18:48:34.467870 1928 ts_tablet_manager.cc:1905] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:34.467798 1845 consensus_queue.cc:260] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.468637 1845 ts_tablet_manager.cc:1428] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:34.475605 1928 ts_tablet_manager.cc:1918] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:34.475768 1928 log.cc:1196] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/752d0eee1f2e485ea7a948460555bccf
I20260810 18:48:34.476240 1928 ts_tablet_manager.cc:1939] T 752d0eee1f2e485ea7a948460555bccf P bc8744abdf9a4109906540822ab6410f: Deleting consensus metadata
I20260810 18:48:34.477865 949 catalog_manager.cc:4841] TS bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217): tablet 752d0eee1f2e485ea7a948460555bccf (table test-table [id=e0f7606befc844d19cdba7f14cc3270f]) successfully deleted
I20260810 18:48:34.686707 1854 raft_consensus.cc:482] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:34.687041 1854 raft_consensus.cc:504] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.688761 1854 leader_election.cc:288] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931)
I20260810 18:48:34.697747 1853 raft_consensus.cc:482] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:34.698230 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.689296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:32819 (local address 127.29.174.193:43217)
0810 18:48:34.689915 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.689922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.690487 (+ 565us) server_negotiation.cc:407] Connection header received
0810 18:48:34.690747 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.690752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.690829 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.690901 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.691879 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.692584 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.695009 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.696743 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.697013 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.697105 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.697631 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.697775 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260810 18:48:34.698009 1853 raft_consensus.cc:504] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.698652 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.689199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:42047 (local address 127.29.174.194:43931)
0810 18:48:34.689412 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:34.689418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:34.690092 (+ 674us) server_negotiation.cc:407] Connection header received
0810 18:48:34.690367 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:34.690373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:34.690441 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:34.690529 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:34.691017 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.691796 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.695133 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.697139 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.697325 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.697402 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:34.698070 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 18:48:34.698206 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:48:34.699069 1853 leader_election.cc:288] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.699862 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.688967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.29.174.195:42047)
0810 18:48:34.689869 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.690005 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.690203 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.690576 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.690581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.690609 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.690865 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.690874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.691924 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.691929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.694899 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.694916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.697556 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.697565 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.697866 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.697900 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.698906 (+ 1006us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.699053 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":52,"thread_start_us":640,"threads_started":1}
I20260810 18:48:34.700253 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb" is_pre_election: true
I20260810 18:48:34.699795 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50" is_pre_election: true
I20260810 18:48:34.700564 1486 raft_consensus.cc:2454] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 0.
I20260810 18:48:34.700587 1721 raft_consensus.cc:2454] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 0.
I20260810 18:48:34.700412 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:34.688983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.29.174.195:32819)
0810 18:48:34.690382 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:34.690401 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:34.690547 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:34.691114 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:34.691119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:34.691145 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:34.691677 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.691694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.692719 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.692723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.694854 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:34.694867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:34.697014 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:34.697022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:34.697372 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:34.697418 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:48:34.698503 (+ 1085us) client_negotiation.cc:240] Negotiation successful
0810 18:48:34.698651 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":237,"thread_start_us":735,"threads_started":1}
I20260810 18:48:34.701428 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb" is_pre_election: true
I20260810 18:48:34.701745 1486 raft_consensus.cc:2454] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 0.
I20260810 18:48:34.701805 1111 leader_election.cc:302] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 790b29ce942540f4a50a3997dd93ff50, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:34.701870 1220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f" is_pre_election: true
I20260810 18:48:34.702175 1853 raft_consensus.cc:2781] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:34.702406 1220 raft_consensus.cc:2454] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 0.
I20260810 18:48:34.702509 1608 leader_election.cc:302] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, 790b29ce942540f4a50a3997dd93ff50; no voters:
I20260810 18:48:34.702796 1853 raft_consensus.cc:482] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:34.702956 1853 raft_consensus.cc:3037] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.703194 1854 raft_consensus.cc:2781] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:34.703387 1854 raft_consensus.cc:482] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:34.703495 1854 raft_consensus.cc:3037] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.705813 1853 raft_consensus.cc:504] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.706672 1853 leader_election.cc:288] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 election: Requested vote from peers 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:34.706749 1854 raft_consensus.cc:504] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.707463 1721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50"
I20260810 18:48:34.707772 1721 raft_consensus.cc:2379] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bc8744abdf9a4109906540822ab6410f in current term 1: Already voted for candidate 790b29ce942540f4a50a3997dd93ff50 in this term.
I20260810 18:48:34.707950 1220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f"
I20260810 18:48:34.708436 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "bc8744abdf9a4109906540822ab6410f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb"
I20260810 18:48:34.708504 1220 raft_consensus.cc:2379] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 790b29ce942540f4a50a3997dd93ff50 in current term 1: Already voted for candidate bc8744abdf9a4109906540822ab6410f in this term.
I20260810 18:48:34.708622 1486 raft_consensus.cc:3037] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:34.708804 1854 leader_election.cc:288] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 election: Requested vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931)
I20260810 18:48:34.709385 1485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35428583799b4292844efd8fed55d003" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb"
I20260810 18:48:34.710091 1608 leader_election.cc:302] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 790b29ce942540f4a50a3997dd93ff50; no voters: 714bae5339f44945bc4ef2d374563cbb, bc8744abdf9a4109906540822ab6410f
I20260810 18:48:34.710486 1854 raft_consensus.cc:2726] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 18:48:34.712206 1486 raft_consensus.cc:2454] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc8744abdf9a4109906540822ab6410f in term 1.
I20260810 18:48:34.712613 1111 leader_election.cc:302] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, bc8744abdf9a4109906540822ab6410f; no voters: 790b29ce942540f4a50a3997dd93ff50
I20260810 18:48:34.712918 1853 raft_consensus.cc:2781] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:34.713138 1853 raft_consensus.cc:686] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 LEADER]: Becoming Leader. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Running, Role: LEADER
I20260810 18:48:34.713506 1853 consensus_queue.cc:237] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:34.716740 970 catalog_manager.cc:5495] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc8744abdf9a4109906540822ab6410f (127.29.174.193). New cstate: current_term: 1 leader_uuid: "bc8744abdf9a4109906540822ab6410f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:34.741937 1721 raft_consensus.cc:1261] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Refusing update from remote peer bc8744abdf9a4109906540822ab6410f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.742708 1486 raft_consensus.cc:1261] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Refusing update from remote peer bc8744abdf9a4109906540822ab6410f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:34.743257 1853 consensus_queue.cc:1035] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [LEADER]: Connected to new peer: Peer: permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:34.743569 1860 consensus_queue.cc:1035] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [LEADER]: Connected to new peer: Peer: permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:48:34.764377 1891 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:48:36.363677 969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.363963 969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.368887 969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Sending DeleteTablet for 3 replicas of tablet 35428583799b4292844efd8fed55d003
I20260810 18:48:36.369712 1440 tablet_service.cc:1514] Processing DeleteTablet for tablet 35428583799b4292844efd8fed55d003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36812
I20260810 18:48:36.369808 1178 tablet_service.cc:1514] Processing DeleteTablet for tablet 35428583799b4292844efd8fed55d003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:55336
I20260810 18:48:36.369964 1688 tablet_service.cc:1514] Processing DeleteTablet for tablet 35428583799b4292844efd8fed55d003 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:47236
I20260810 18:48:36.383121 1990 tablet_replica.cc:330] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: stopping tablet replica
I20260810 18:48:36.383186 1982 tablet_replica.cc:330] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: stopping tablet replica
I20260810 18:48:36.383456 1990 raft_consensus.cc:2229] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.383478 1982 raft_consensus.cc:2229] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.383858 1982 raft_consensus.cc:2258] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.383886 1990 raft_consensus.cc:2258] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.384979 1982 ts_tablet_manager.cc:1905] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.385123 1988 tablet_replica.cc:330] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: stopping tablet replica
I20260810 18:48:36.385396 1988 raft_consensus.cc:2229] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:36.385999 1990 ts_tablet_manager.cc:1905] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.386127 1988 raft_consensus.cc:2258] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.387585 1988 ts_tablet_manager.cc:1905] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.393399 1982 ts_tablet_manager.cc:1918] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:36.393592 1982 log.cc:1196] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/35428583799b4292844efd8fed55d003
I20260810 18:48:36.393603 1988 ts_tablet_manager.cc:1918] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:36.393779 1988 log.cc:1196] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/35428583799b4292844efd8fed55d003
I20260810 18:48:36.393752 1990 ts_tablet_manager.cc:1918] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:36.393951 1990 log.cc:1196] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/35428583799b4292844efd8fed55d003
I20260810 18:48:36.394245 1982 ts_tablet_manager.cc:1939] T 35428583799b4292844efd8fed55d003 P 714bae5339f44945bc4ef2d374563cbb: Deleting consensus metadata
I20260810 18:48:36.394279 1988 ts_tablet_manager.cc:1939] T 35428583799b4292844efd8fed55d003 P bc8744abdf9a4109906540822ab6410f: Deleting consensus metadata
I20260810 18:48:36.394604 1990 ts_tablet_manager.cc:1939] T 35428583799b4292844efd8fed55d003 P 790b29ce942540f4a50a3997dd93ff50: Deleting consensus metadata
I20260810 18:48:36.395399 949 catalog_manager.cc:4841] TS bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217): tablet 35428583799b4292844efd8fed55d003 (table test-table [id=d783e53206a64885bc3a9ac12493bc7e]) successfully deleted
I20260810 18:48:36.395365 951 catalog_manager.cc:4841] TS 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931): tablet 35428583799b4292844efd8fed55d003 (table test-table [id=d783e53206a64885bc3a9ac12493bc7e]) successfully deleted
I20260810 18:48:36.395722 951 catalog_manager.cc:4841] TS 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111): tablet 35428583799b4292844efd8fed55d003 (table test-table [id=d783e53206a64885bc3a9ac12493bc7e]) successfully deleted
Aug 10 18:48:36 dist-test-slave-6zbq krb5kdc[903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387716, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:48:36 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387716, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:36.442377 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.425042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.29.174.254:39425)
0810 18:48:36.425905 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.425910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.425947 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:48:36.425997 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.426001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.426056 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.426143 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:36.427502 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.428420 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.430090 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.430662 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.430767 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.435770 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:36.435776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:36.435779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:36.435808 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:36.439629 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:36.440336 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:36.440340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:36.440343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:36.440522 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:36.440824 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:36.440827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:36.440829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:36.441199 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:36.441423 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.441582 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.441770 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":495,"threads_started":1}
I20260810 18:48:36.445724 969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:36.446403 969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:36.452860 1178 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b (DEFAULT_TABLE table=test-table [id=4b79a8283fe34a86b2553aa391e170ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.452941 1440 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b (DEFAULT_TABLE table=test-table [id=4b79a8283fe34a86b2553aa391e170ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.453382 1178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3ffd2632c64af7b1a09adf76b1a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.453352 1688 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b (DEFAULT_TABLE table=test-table [id=4b79a8283fe34a86b2553aa391e170ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.453528 1440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3ffd2632c64af7b1a09adf76b1a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.453817 1688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3ffd2632c64af7b1a09adf76b1a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.459522 2005 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Bootstrap starting.
I20260810 18:48:36.461894 2006 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Bootstrap starting.
I20260810 18:48:36.462252 2005 tablet_bootstrap.cc:654] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.464753 2006 tablet_bootstrap.cc:654] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.465399 2005 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: No bootstrap required, opened a new log
I20260810 18:48:36.465582 2005 ts_tablet_manager.cc:1397] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:48:36.465813 2007 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Bootstrap starting.
I20260810 18:48:36.466886 2005 raft_consensus.cc:348] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.467159 2005 raft_consensus.cc:374] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.467401 2005 raft_consensus.cc:729] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.467753 2005 consensus_queue.cc:260] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.468515 2005 ts_tablet_manager.cc:1428] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:36.468802 2007 tablet_bootstrap.cc:654] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.468910 2006 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: No bootstrap required, opened a new log
I20260810 18:48:36.470038 2006 ts_tablet_manager.cc:1397] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 18:48:36.471161 2006 raft_consensus.cc:348] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.471567 2006 raft_consensus.cc:374] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.471685 2006 raft_consensus.cc:729] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.471925 2007 tablet_bootstrap.cc:492] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: No bootstrap required, opened a new log
I20260810 18:48:36.472060 2007 ts_tablet_manager.cc:1397] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260810 18:48:36.471978 2006 consensus_queue.cc:260] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.473067 2007 raft_consensus.cc:348] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.473367 2007 raft_consensus.cc:374] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.473484 2007 raft_consensus.cc:729] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.473775 2007 consensus_queue.cc:260] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.474426 2007 ts_tablet_manager.cc:1428] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:36.475850 2006 ts_tablet_manager.cc:1428] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:48:36.569679 2012 raft_consensus.cc:482] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:36.570114 2012 raft_consensus.cc:504] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.571123 2012 leader_election.cc:288] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931)
I20260810 18:48:36.571856 1220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3ffd2632c64af7b1a09adf76b1a27b" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f" is_pre_election: true
I20260810 18:48:36.571811 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3ffd2632c64af7b1a09adf76b1a27b" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb" is_pre_election: true
I20260810 18:48:36.572212 1486 raft_consensus.cc:2454] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 0.
I20260810 18:48:36.572206 1220 raft_consensus.cc:2454] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 0.
I20260810 18:48:36.572814 1606 leader_election.cc:302] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, 790b29ce942540f4a50a3997dd93ff50, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:36.573238 2012 raft_consensus.cc:2781] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:36.573421 2012 raft_consensus.cc:482] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:36.573580 2012 raft_consensus.cc:3037] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.576679 2012 raft_consensus.cc:504] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.577344 2012 leader_election.cc:288] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 election: Requested vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931)
I20260810 18:48:36.577852 1220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3ffd2632c64af7b1a09adf76b1a27b" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f"
I20260810 18:48:36.577904 1486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3ffd2632c64af7b1a09adf76b1a27b" candidate_uuid: "790b29ce942540f4a50a3997dd93ff50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "714bae5339f44945bc4ef2d374563cbb"
I20260810 18:48:36.578094 1486 raft_consensus.cc:3037] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.578127 1220 raft_consensus.cc:3037] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.580621 1486 raft_consensus.cc:2454] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 1.
I20260810 18:48:36.580999 1220 raft_consensus.cc:2454] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 790b29ce942540f4a50a3997dd93ff50 in term 1.
I20260810 18:48:36.581203 1608 leader_election.cc:302] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, 790b29ce942540f4a50a3997dd93ff50; no voters:
I20260810 18:48:36.581604 2012 raft_consensus.cc:2781] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:36.582149 2012 raft_consensus.cc:686] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 1 LEADER]: Becoming Leader. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Running, Role: LEADER
I20260810 18:48:36.583000 2012 consensus_queue.cc:237] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.586849 969 catalog_manager.cc:5495] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "790b29ce942540f4a50a3997dd93ff50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:36.635385 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.627284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.29.174.195:45111)
0810 18:48:36.627983 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.627988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.628007 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:36.628087 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.628090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.628172 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.628307 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:36.628703 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.629949 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.631890 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.633286 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.633602 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.634136 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.634778 (+ 642us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.634889 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.634988 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.635066 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":443,"threads_started":1}
I20260810 18:48:36.654749 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.647242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35035 (local address 127.29.174.194:43931)
0810 18:48:36.648558 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.648563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.648598 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:48:36.648657 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.648660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.648726 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.648836 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:36.649722 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.650728 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.652603 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.653160 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.653279 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.653836 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.653940 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.654019 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.654152 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.654235 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":964,"threads_started":1}
I20260810 18:48:36.654860 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.647083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.29.174.195:35035)
0810 18:48:36.647948 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.647968 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.648072 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.648875 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.648880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.648894 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:36.649445 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.649459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.650891 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.650895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.652466 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.652479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.653371 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.653378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.653537 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.653615 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.654042 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.654053 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.654122 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.654255 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":557,"threads_started":1}
I20260810 18:48:36.663301 1486 raft_consensus.cc:1261] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Refusing update from remote peer 790b29ce942540f4a50a3997dd93ff50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.663452 1220 raft_consensus.cc:1261] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Refusing update from remote peer 790b29ce942540f4a50a3997dd93ff50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.664088 2016 consensus_queue.cc:1035] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:36.664700 2012 consensus_queue.cc:1035] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:36.693949 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.688094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:46231 (local address 127.29.174.195:45111)
0810 18:48:36.688271 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.688278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.688688 (+ 410us) server_negotiation.cc:407] Connection header received
0810 18:48:36.688906 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.688912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.688975 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.689044 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:36.689473 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.690267 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.692170 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.692644 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.692758 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.693313 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.693407 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.693480 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.693617 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.693694 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:48:36.694245 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.687963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:45111 (local address 127.29.174.194:46231)
0810 18:48:36.688594 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.688612 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.688778 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.689079 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.689084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.689095 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:36.689339 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.689352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.690407 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.690411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.691952 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.691966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.692825 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.692833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.693121 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.693161 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.693497 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.693509 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.693580 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.693708 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":346,"threads_started":1}
I20260810 18:48:36.732924 970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.733199 970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:36.736340 970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Sending DeleteTablet for 3 replicas of tablet 2b3ffd2632c64af7b1a09adf76b1a27b
I20260810 18:48:36.737115 1178 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:55336
I20260810 18:48:36.737291 1440 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36812
I20260810 18:48:36.737290 1688 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3ffd2632c64af7b1a09adf76b1a27b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:36 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:47236
I20260810 18:48:36.737406 1988 tablet_replica.cc:330] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: stopping tablet replica
I20260810 18:48:36.737607 1990 tablet_replica.cc:330] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: stopping tablet replica
I20260810 18:48:36.737613 1988 raft_consensus.cc:2229] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.737783 1982 tablet_replica.cc:330] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: stopping tablet replica
I20260810 18:48:36.737886 1990 raft_consensus.cc:2229] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:36.737887 1988 raft_consensus.cc:2258] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.738049 1982 raft_consensus.cc:2229] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:36.738262 1982 raft_consensus.cc:2258] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.738483 1990 raft_consensus.cc:2258] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:36.739078 1988 ts_tablet_manager.cc:1905] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.739620 1990 ts_tablet_manager.cc:1905] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.740265 970 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:36.740868 970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:36.742784 1982 ts_tablet_manager.cc:1905] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:36.746659 1988 ts_tablet_manager.cc:1918] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:36.746863 1988 log.cc:1196] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/2b3ffd2632c64af7b1a09adf76b1a27b
I20260810 18:48:36.747371 1988 ts_tablet_manager.cc:1939] T 2b3ffd2632c64af7b1a09adf76b1a27b P bc8744abdf9a4109906540822ab6410f: Deleting consensus metadata
I20260810 18:48:36.748623 949 catalog_manager.cc:4841] TS bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217): tablet 2b3ffd2632c64af7b1a09adf76b1a27b (table test-table [id=4b79a8283fe34a86b2553aa391e170ac]) successfully deleted
I20260810 18:48:36.751081 1178 tablet_service.cc:1467] Processing CreateTablet for tablet ae237aedc5c24352972cc2863d86ed41 (DEFAULT_TABLE table=test-table [id=70fd1ac403eb4e5dbfd47b163b24a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.751168 1440 tablet_service.cc:1467] Processing CreateTablet for tablet ae237aedc5c24352972cc2863d86ed41 (DEFAULT_TABLE table=test-table [id=70fd1ac403eb4e5dbfd47b163b24a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.751186 1688 tablet_service.cc:1467] Processing CreateTablet for tablet ae237aedc5c24352972cc2863d86ed41 (DEFAULT_TABLE table=test-table [id=70fd1ac403eb4e5dbfd47b163b24a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:36.751487 1178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae237aedc5c24352972cc2863d86ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.751677 1688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae237aedc5c24352972cc2863d86ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.752545 1440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae237aedc5c24352972cc2863d86ed41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:36.754689 1990 ts_tablet_manager.cc:1918] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:36.754946 1990 log.cc:1196] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/2b3ffd2632c64af7b1a09adf76b1a27b
I20260810 18:48:36.755471 1990 ts_tablet_manager.cc:1939] T 2b3ffd2632c64af7b1a09adf76b1a27b P 790b29ce942540f4a50a3997dd93ff50: Deleting consensus metadata
I20260810 18:48:36.756512 951 catalog_manager.cc:4841] TS 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111): tablet 2b3ffd2632c64af7b1a09adf76b1a27b (table test-table [id=4b79a8283fe34a86b2553aa391e170ac]) successfully deleted
I20260810 18:48:36.758396 1982 ts_tablet_manager.cc:1918] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:48:36.758903 1982 log.cc:1196] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/2b3ffd2632c64af7b1a09adf76b1a27b
I20260810 18:48:36.759075 2007 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: Bootstrap starting.
I20260810 18:48:36.760564 2005 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Bootstrap starting.
I20260810 18:48:36.760747 1982 ts_tablet_manager.cc:1939] T 2b3ffd2632c64af7b1a09adf76b1a27b P 714bae5339f44945bc4ef2d374563cbb: Deleting consensus metadata
I20260810 18:48:36.761453 2007 tablet_bootstrap.cc:654] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.762794 2005 tablet_bootstrap.cc:654] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.762989 2006 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Bootstrap starting.
I20260810 18:48:36.764392 951 catalog_manager.cc:4841] TS 714bae5339f44945bc4ef2d374563cbb (127.29.174.194:43931): tablet 2b3ffd2632c64af7b1a09adf76b1a27b (table test-table [id=4b79a8283fe34a86b2553aa391e170ac]) successfully deleted
I20260810 18:48:36.765473 2006 tablet_bootstrap.cc:654] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:36.766259 2007 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: No bootstrap required, opened a new log
I20260810 18:48:36.766582 2007 ts_tablet_manager.cc:1397] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260810 18:48:36.767786 2007 raft_consensus.cc:348] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.769349 2007 raft_consensus.cc:374] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.769512 2007 raft_consensus.cc:729] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc8744abdf9a4109906540822ab6410f, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.769861 2007 consensus_queue.cc:260] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.770668 2007 ts_tablet_manager.cc:1428] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260810 18:48:36.773042 2005 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: No bootstrap required, opened a new log
I20260810 18:48:36.773270 2005 ts_tablet_manager.cc:1397] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260810 18:48:36.774456 2005 raft_consensus.cc:348] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.774837 2005 raft_consensus.cc:374] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.774957 2005 raft_consensus.cc:729] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.775243 2005 consensus_queue.cc:260] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.776556 2005 ts_tablet_manager.cc:1428] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:36.778167 2006 tablet_bootstrap.cc:492] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: No bootstrap required, opened a new log
I20260810 18:48:36.778383 2006 ts_tablet_manager.cc:1397] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260810 18:48:36.780577 2006 raft_consensus.cc:348] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.780933 2006 raft_consensus.cc:374] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:36.781111 2006 raft_consensus.cc:729] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790b29ce942540f4a50a3997dd93ff50, State: Initialized, Role: FOLLOWER
I20260810 18:48:36.781546 2006 consensus_queue.cc:260] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.782295 2006 ts_tablet_manager.cc:1428] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 18:48:36.788513 1908 raft_consensus.cc:482] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:36.788658 1908 raft_consensus.cc:504] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.789443 1908 leader_election.cc:288] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:36.789841 1221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae237aedc5c24352972cc2863d86ed41" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f" is_pre_election: true
I20260810 18:48:36.789870 1719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae237aedc5c24352972cc2863d86ed41" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50" is_pre_election: true
I20260810 18:48:36.790094 1221 raft_consensus.cc:2454] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 0.
I20260810 18:48:36.790243 1719 raft_consensus.cc:2454] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 0.
I20260810 18:48:36.790567 1347 leader_election.cc:302] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:36.790876 1908 raft_consensus.cc:2781] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:36.791002 1908 raft_consensus.cc:482] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:36.791097 1908 raft_consensus.cc:3037] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.794716 1908 raft_consensus.cc:504] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.795544 1908 leader_election.cc:288] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 election: Requested vote from peers bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217), 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111)
I20260810 18:48:36.796284 1221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae237aedc5c24352972cc2863d86ed41" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc8744abdf9a4109906540822ab6410f"
I20260810 18:48:36.796398 1719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae237aedc5c24352972cc2863d86ed41" candidate_uuid: "714bae5339f44945bc4ef2d374563cbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "790b29ce942540f4a50a3997dd93ff50"
I20260810 18:48:36.796536 1221 raft_consensus.cc:3037] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.796659 1719 raft_consensus.cc:3037] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:36.800592 1719 raft_consensus.cc:2454] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 1.
I20260810 18:48:36.800664 1221 raft_consensus.cc:2454] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 714bae5339f44945bc4ef2d374563cbb in term 1.
I20260810 18:48:36.801090 1347 leader_election.cc:302] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 714bae5339f44945bc4ef2d374563cbb, bc8744abdf9a4109906540822ab6410f; no voters:
I20260810 18:48:36.801476 1908 raft_consensus.cc:2781] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:36.801668 1908 raft_consensus.cc:686] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 1 LEADER]: Becoming Leader. State: Replica: 714bae5339f44945bc4ef2d374563cbb, State: Running, Role: LEADER
I20260810 18:48:36.802124 1908 consensus_queue.cc:237] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } }
I20260810 18:48:36.805371 969 catalog_manager.cc:5495] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 714bae5339f44945bc4ef2d374563cbb (127.29.174.194). New cstate: current_term: 1 leader_uuid: "714bae5339f44945bc4ef2d374563cbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "714bae5339f44945bc4ef2d374563cbb" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 } health_report { overall_health: UNKNOWN } } }
I20260810 18:48:36.836006 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.825558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53914 (local address 127.29.174.194:43931)
0810 18:48:36.825722 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.825728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.825748 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:48:36.826108 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.826114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.826196 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.826289 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:36.826869 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.828295 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.832809 (+ 4514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.833688 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.833818 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.834575 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.834707 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.834793 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.835564 (+ 771us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.835661 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:48:36.858973 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.854130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:58813 (local address 127.29.174.194:43931)
0810 18:48:36.854265 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:36.854269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:36.854288 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:36.854408 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:36.854414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:36.854481 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:36.854560 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:36.854917 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.855784 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.857304 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.857748 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.857856 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.858227 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.858319 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.858434 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:36.858546 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 18:48:36.858611 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:48:36.859051 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:36.853889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.29.174.194:58813)
0810 18:48:36.854091 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:36.854116 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:36.854239 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:36.854586 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:36.854590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:36.854601 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:48:36.854775 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.854785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.855943 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.855949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.857181 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:36.857193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:36.857878 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:36.857884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:36.858049 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:36.858084 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:48:36.858432 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:48:36.858443 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:48:36.858516 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:48:36.858645 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:48:36.867534 1719 raft_consensus.cc:1261] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Refusing update from remote peer 714bae5339f44945bc4ef2d374563cbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.868148 1221 raft_consensus.cc:1261] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Refusing update from remote peer 714bae5339f44945bc4ef2d374563cbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:36.868460 1908 consensus_queue.cc:1035] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "790b29ce942540f4a50a3997dd93ff50" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:36.868940 1952 consensus_queue.cc:1035] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc8744abdf9a4109906540822ab6410f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:48:36.902068 1945 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:48:38.497349 970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:38.497663 970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:38.503378 970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ce6726a946e42d2a6b875371a5d0714: Sending DeleteTablet for 3 replicas of tablet ae237aedc5c24352972cc2863d86ed41
I20260810 18:48:38.504310 1178 tablet_service.cc:1514] Processing DeleteTablet for tablet ae237aedc5c24352972cc2863d86ed41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:38 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:55336
I20260810 18:48:38.504478 1440 tablet_service.cc:1514] Processing DeleteTablet for tablet ae237aedc5c24352972cc2863d86ed41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:38 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36812
I20260810 18:48:38.504596 1688 tablet_service.cc:1514] Processing DeleteTablet for tablet ae237aedc5c24352972cc2863d86ed41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:38 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:47236
I20260810 18:48:38.505288 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1083
I20260810 18:48:38.516800 2081 tablet_replica.cc:330] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: stopping tablet replica
I20260810 18:48:38.517385 2084 tablet_replica.cc:330] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: stopping tablet replica
I20260810 18:48:38.517712 2084 raft_consensus.cc:2229] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:38.518049 2081 raft_consensus.cc:2229] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:38.518440 2084 raft_consensus.cc:2258] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:38.518826 2081 raft_consensus.cc:2258] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:38.519167 2082 tablet_replica.cc:330] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: stopping tablet replica
I20260810 18:48:38.519444 2082 raft_consensus.cc:2229] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:38.520021 2082 raft_consensus.cc:2258] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:38.520289 2084 ts_tablet_manager.cc:1905] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:38.520709 2081 ts_tablet_manager.cc:1905] T ae237aedc5c24352972cc2863d86ed41 P bc8744abdf9a4109906540822ab6410f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:38.521502 2082 ts_tablet_manager.cc:1905] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 18:48:38 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387716, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:38.541636 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.507426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.29.174.193:43217)
0810 18:48:38.518052 (+ 10626us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:38.518058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:38.519027 (+ 969us) server_negotiation.cc:407] Connection header received
0810 18:48:38.519117 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:38.519122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:38.519228 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:38.519347 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:38.521384 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.523816 (+ 2432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.525006 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.525532 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.525660 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:38.534821 (+ 9161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:38.534836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:38.534852 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:38.534890 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:38.538261 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.539045 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.539051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.539054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.539262 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.539675 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.539679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.539681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.540110 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:38.540498 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:38.540773 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 18:48:38.541051 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10421,"thread_start_us":10197,"threads_started":1}
W20260810 18:48:38.542333 1304 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52238
W20260810 18:48:38.543000 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1083: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:38.543192 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1083
I20260810 18:48:38.582089 2082 ts_tablet_manager.cc:1918] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:38.582199 2084 ts_tablet_manager.cc:1918] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:48:38.582332 2082 log.cc:1196] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/ae237aedc5c24352972cc2863d86ed41
I20260810 18:48:38.582487 2084 log.cc:1196] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/ae237aedc5c24352972cc2863d86ed41
I20260810 18:48:38.583081 2082 ts_tablet_manager.cc:1939] T ae237aedc5c24352972cc2863d86ed41 P 714bae5339f44945bc4ef2d374563cbb: Deleting consensus metadata
I20260810 18:48:38.583223 2084 ts_tablet_manager.cc:1939] T ae237aedc5c24352972cc2863d86ed41 P 790b29ce942540f4a50a3997dd93ff50: Deleting consensus metadata
W20260810 18:48:38.599611 949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:43217 (error 108)
I20260810 18:48:38.600521 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1318
I20260810 18:48:38.602564 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.601356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:52240)
0810 18:48:38.602126 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.602284 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":503,"threads_started":1}
W20260810 18:48:38.602921 949 catalog_manager.cc:4584] TS bc8744abdf9a4109906540822ab6410f (127.29.174.193:43217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae237aedc5c24352972cc2863d86ed41: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Aug 10 18:48:38 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387716, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:38.618487 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.601356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.29.174.194:43931)
0810 18:48:38.602168 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:38.602174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:38.602194 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:48:38.602258 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:38.602264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:38.602418 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:38.602542 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:38.604823 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.605947 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.607582 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.608101 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.608228 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:38.613305 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:38.613321 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:38.613338 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:38.613396 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:38.616181 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.616771 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.616778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.616780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.616933 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.617226 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.617230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.617233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.617578 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:38.617792 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:38.618007 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 18:48:38.618180 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":469,"threads_started":1}
W20260810 18:48:38.619087 1558 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53920
W20260810 18:48:38.619685 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1318: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:38.619828 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1318
I20260810 18:48:38.656543 951 catalog_manager.cc:4841] TS 790b29ce942540f4a50a3997dd93ff50 (127.29.174.195:45111): tablet ae237aedc5c24352972cc2863d86ed41 (table test-table [id=70fd1ac403eb4e5dbfd47b163b24a266]) successfully deleted
I20260810 18:48:38.656627 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.656116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:52242)
0810 18:48:38.656342 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.656456 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 18:48:38.657665 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.657238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:52250)
0810 18:48:38.657398 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.657518 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 18:48:38.681402 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1575
I20260810 18:48:38.682391 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.681873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:53924)
0810 18:48:38.682082 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.682177 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.194:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
Aug 10 18:48:38 dist-test-slave-6zbq krb5kdc[903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387716, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:38.696564 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.682125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.29.174.195:45111)
0810 18:48:38.682690 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:38.682696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:38.682715 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:38.682777 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:38.682782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:38.682848 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:38.682940 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:38.684415 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.685184 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.686413 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:38.686831 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:38.686942 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:38.691085 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:38.691100 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:38.691117 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:38.691149 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:38.693869 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.694531 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.694539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.694541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.694672 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:38.695041 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:38.695045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:38.695047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:38.695417 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:38.695722 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:38.696016 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 18:48:38.696262 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":292,"threads_started":1}
W20260810 18:48:38.697201 1800 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51446
W20260810 18:48:38.697759 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1575: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:38.697916 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1575
I20260810 18:48:38.714938 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.714412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:53940)
0810 18:48:38.714635 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.714742 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.194:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:48:38.715960 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.715583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:53948)
0810 18:48:38.715748 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.715828 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.194:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 18:48:38.718209 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.717887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:52260)
0810 18:48:38.718025 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.718088 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:48:38.719077 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.718778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43217 (local address 127.0.0.1:52274)
0810 18:48:38.718918 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.718984 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 18:48:38.725293 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 923
I20260810 18:48:38.726276 1042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:38.846107 2091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:38.845224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43931 (local address 127.0.0.1:53954)
0810 18:48:38.845722 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:38.845843 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.194:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269}
I20260810 18:48:38.846522 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 923
2026-08-10T18:48:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8573 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:40 dist-test-slave-6zbq krb5kdc[2101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:40 dist-test-slave-6zbq krb5kdc[2101](info): set up 2 sockets
Aug 10 18:48:40 dist-test-slave-6zbq krb5kdc[2101](info): commencing operation
krb5kdc: starting...
W20260810 18:48:41.288571 30395 mini_kdc.cc:121] Time spent starting KDC: real 2.394s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:41 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:41.478092 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46779
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:46779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:41.786840 2125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:41.787125 2125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:41.787261 2125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:41.794268 2125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:41.794389 2125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:41.794467 2125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:41.794544 2125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:41.794603 2125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:41.803988 2125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:46779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:41.805193 2125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:41.806439 2125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:41.817495 2132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:41.817464 2134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:41.817600 2137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:41.818166 2125 server_base.cc:1034] running on GCE node
I20260810 18:48:41.818921 2125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:41.820165 2125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:41.821365 2125 hybrid_clock.cc:648] HybridClock initialized: now 1786387721821317 us; error 37 us; skew 500 ppm
I20260810 18:48:41.821679 2125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:41 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:41.824503 2125 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:41.825819 2125 webserver.cc:460] Webserver started at http://127.29.174.254:34909/ using document root <none> and password file <none>
I20260810 18:48:41.826502 2125 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:41.826622 2125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:41.826905 2125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:41.829696 2125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "bd86c33b93344aa597be66dd9dabadb8"
format_stamp: "Formatted at 2026-08-10 18:48:41 on dist-test-slave-6zbq"
server_key: "f0d9c536f47eb6775992e9c93f79213c"
server_key_iv: "3ae1a996d5bb696a03646d71d44ce7a3"
server_key_version: "encryptionkey@0"
I20260810 18:48:41.830492 2125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "bd86c33b93344aa597be66dd9dabadb8"
format_stamp: "Formatted at 2026-08-10 18:48:41 on dist-test-slave-6zbq"
server_key: "f0d9c536f47eb6775992e9c93f79213c"
server_key_iv: "3ae1a996d5bb696a03646d71d44ce7a3"
server_key_version: "encryptionkey@0"
I20260810 18:48:41.835292 2125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:41.838524 2149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:41.839704 2125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:48:41.839874 2125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "bd86c33b93344aa597be66dd9dabadb8"
format_stamp: "Formatted at 2026-08-10 18:48:41 on dist-test-slave-6zbq"
server_key: "f0d9c536f47eb6775992e9c93f79213c"
server_key_iv: "3ae1a996d5bb696a03646d71d44ce7a3"
server_key_version: "encryptionkey@0"
I20260810 18:48:41.840044 2125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:41.871366 2125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:41.875269 2125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:41.875514 2125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:41.892151 2125 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:46779
I20260810 18:48:41.892174 2231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:46779 every 8 connection(s)
I20260810 18:48:41.893445 2125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:41.894516 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2125
I20260810 18:48:41.894762 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:41.895541 30395 external_mini_cluster.cc:1428] Setting key daf3ef1cde549c5d73b8c3e315530b16
I20260810 18:48:41.897949 2233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:41.907899 2233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8: Bootstrap starting.
Aug 10 18:48:41 dist-test-slave-6zbq krb5kdc[2101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:41.911212 2233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:41.912449 2233 log.cc:824] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:41.915689 2233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8: No bootstrap required, opened a new log
I20260810 18:48:41.917372 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:41.897544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45180 (local address 127.29.174.254:46779)
0810 18:48:41.898331 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:41.898367 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:41.898414 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:48:41.900003 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:41.900024 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:41.900386 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:41.900719 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:41.902162 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:41.903503 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:41.904737 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:41.905416 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:41.905593 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:41.909335 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:41.909347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:41.909368 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:41.909415 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:41.913043 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:41.913854 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:41.913862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:41.913870 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:41.914065 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:41.914405 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:41.914409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:41.914413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:41.915041 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:41.915321 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:41.915613 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 18:48:41.916028 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":337,"threads_started":1}
I20260810 18:48:41.923842 2233 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } }
I20260810 18:48:41.924160 2233 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:41.924232 2233 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd86c33b93344aa597be66dd9dabadb8, State: Initialized, Role: FOLLOWER
I20260810 18:48:41.924666 2233 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } }
I20260810 18:48:41.924921 2233 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:41.925067 2233 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:41.925259 2233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:41.927356 2233 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } }
I20260810 18:48:41.927790 2233 leader_election.cc:302] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd86c33b93344aa597be66dd9dabadb8; no voters:
I20260810 18:48:41.928305 2233 leader_election.cc:288] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:41.928414 2244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:41.929436 2244 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [term 1 LEADER]: Becoming Leader. State: Replica: bd86c33b93344aa597be66dd9dabadb8, State: Running, Role: LEADER
I20260810 18:48:41.929961 2244 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } }
I20260810 18:48:41.930169 2233 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:41.932533 2247 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd86c33b93344aa597be66dd9dabadb8. Latest consensus state: current_term: 1 leader_uuid: "bd86c33b93344aa597be66dd9dabadb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } } }
I20260810 18:48:41.932982 2247 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:41.932559 2246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd86c33b93344aa597be66dd9dabadb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd86c33b93344aa597be66dd9dabadb8" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46779 } } }
I20260810 18:48:41.933333 2246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:41.935467 2254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:41.940354 2254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:41.952535 2254 catalog_manager.cc:1347] Generated new cluster ID: bc480e9350304d1e98bd14f3be865775
I20260810 18:48:41.952636 2254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:41.964265 2254 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:41.966001 2254 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:41.975776 2254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd86c33b93344aa597be66dd9dabadb8: Generated new TSK 0
I20260810 18:48:41.976720 2254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:42.028056 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:42.345763 2274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:42.346025 2274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:42.346156 2274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:42.353279 2274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:42.353384 2274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:42.353538 2274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:42.362876 2274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:42.363997 2274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:42.365227 2274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:42.377149 2286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.377264 2287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.377740 2289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:42.378106 2274 server_base.cc:1034] running on GCE node
I20260810 18:48:42.378713 2274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:42.379550 2274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:42.380744 2274 hybrid_clock.cc:648] HybridClock initialized: now 1786387722380698 us; error 42 us; skew 500 ppm
I20260810 18:48:42.381021 2274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:42 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:42.387900 2274 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:42.389161 2274 webserver.cc:460] Webserver started at http://127.29.174.193:37837/ using document root <none> and password file <none>
I20260810 18:48:42.389806 2274 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:42.389920 2274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:42.390218 2274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:42.392997 2274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "f57f3c1e4b954bb88ab585208e75f923"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "2d1e4f83db93be6d28db3958a55b929b"
server_key_iv: "e1eff030d7e829ff53fe233b726b2668"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.393826 2274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "f57f3c1e4b954bb88ab585208e75f923"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "2d1e4f83db93be6d28db3958a55b929b"
server_key_iv: "e1eff030d7e829ff53fe233b726b2668"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.398654 2274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 18:48:42.401816 2306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.403008 2274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:42.403158 2274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "f57f3c1e4b954bb88ab585208e75f923"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "2d1e4f83db93be6d28db3958a55b929b"
server_key_iv: "e1eff030d7e829ff53fe233b726b2668"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.403337 2274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:42.419143 2274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:42.422649 2274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:42.422894 2274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:42.423769 2274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:42.424983 2274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:42.425060 2274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.425175 2274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:42.425239 2274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.456663 2274 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:36553
I20260810 18:48:42.456705 2489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:36553 every 8 connection(s)
I20260810 18:48:42.457883 2274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:42.466833 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2274
I20260810 18:48:42.467057 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:42.467815 30395 external_mini_cluster.cc:1428] Setting key 073465a9f1b9944702f113728f71b8b1
Aug 10 18:48:42 dist-test-slave-6zbq krb5kdc[2101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:42.482843 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.460730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46779 (local address 127.29.174.193:49763)
0810 18:48:42.461878 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:42.461938 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:42.464230 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:42.465040 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:42.465052 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:42.465797 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:42.466609 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.466637 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.468100 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.468105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.469977 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.469992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.471160 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.471166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.471443 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.472164 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:42.472195 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:42.474838 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:42.478571 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.478579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.478594 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.479051 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.479389 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.479393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.479395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.479589 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.480145 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:42.480153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:42.480582 (+ 429us) client_negotiation.cc:769] Sending connection context
0810 18:48:42.480884 (+ 302us) client_negotiation.cc:240] Negotiation successful
0810 18:48:42.481386 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":616,"threads_started":1}
I20260810 18:48:42.483059 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.460184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49763 (local address 127.29.174.254:46779)
0810 18:48:42.461547 (+ 1363us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:42.461554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:42.462059 (+ 505us) server_negotiation.cc:407] Connection header received
0810 18:48:42.464555 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:42.464562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:42.464650 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:42.464784 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:42.466834 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.467911 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.470127 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.470762 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.470881 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.475112 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:42.475122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:42.475127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:42.475165 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:42.478433 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.479171 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.479175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.479178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.479296 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.479702 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.479707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.479711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.480022 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:42.480157 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:42.482528 (+ 2371us) server_negotiation.cc:299] Negotiation successful
0810 18:48:42.482747 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":1023,"threads_started":1}
I20260810 18:48:42.486136 2490 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46779
I20260810 18:48:42.486558 2490 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:42.487396 2490 heartbeater.cc:511] Master 127.29.174.254:46779 requested a full tablet report, sending...
I20260810 18:48:42.489763 2172 ts_manager.cc:194] Registered new tserver with Master: f57f3c1e4b954bb88ab585208e75f923 (127.29.174.193:36553)
I20260810 18:48:42.491812 2172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:49763
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:42.523530 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:42.829291 2500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:42.829525 2500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:42.829620 2500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:42.836534 2500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:42.836632 2500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:42.836741 2500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:42.845837 2500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:42.846894 2500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:42.848037 2500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:42.859670 2510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.859797 2508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:42.860110 2514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:42.860479 2500 server_base.cc:1034] running on GCE node
I20260810 18:48:42.861019 2500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:42.861896 2500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:42.863086 2500 hybrid_clock.cc:648] HybridClock initialized: now 1786387722863045 us; error 28 us; skew 500 ppm
I20260810 18:48:42.863363 2500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:42 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:42.866118 2500 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:42.867426 2500 webserver.cc:460] Webserver started at http://127.29.174.194:43151/ using document root <none> and password file <none>
I20260810 18:48:42.868026 2500 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:42.868137 2500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:42.868412 2500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:42.871073 2500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "f3b21be3e13445dd97450b64bee72b1c"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "87d2606efd430d1d65a575cd226a6b88"
server_key_iv: "fae3f76145f8248327fbaf63fcfe6e97"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.871798 2500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "f3b21be3e13445dd97450b64bee72b1c"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "87d2606efd430d1d65a575cd226a6b88"
server_key_iv: "fae3f76145f8248327fbaf63fcfe6e97"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.876567 2500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:42.879688 2529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.880837 2500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:48:42.880978 2500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "f3b21be3e13445dd97450b64bee72b1c"
format_stamp: "Formatted at 2026-08-10 18:48:42 on dist-test-slave-6zbq"
server_key: "87d2606efd430d1d65a575cd226a6b88"
server_key_iv: "fae3f76145f8248327fbaf63fcfe6e97"
server_key_version: "encryptionkey@0"
I20260810 18:48:42.881152 2500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:42.900692 2500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:42.905190 2500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:42.905472 2500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:42.906466 2500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:42.907994 2500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:42.908078 2500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.908197 2500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:42.908259 2500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:42.938414 2500 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:46211
I20260810 18:48:42.938479 2724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:46211 every 8 connection(s)
I20260810 18:48:42.939610 2500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:42.940053 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2500
I20260810 18:48:42.940231 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:42.940799 30395 external_mini_cluster.cc:1428] Setting key adf84a44d76927374f8f5fe7084041a2
Aug 10 18:48:42 dist-test-slave-6zbq krb5kdc[2101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387722, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:42.959450 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.942259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:55449 (local address 127.29.174.254:46779)
0810 18:48:42.942469 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:42.942475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:42.943741 (+ 1266us) server_negotiation.cc:407] Connection header received
0810 18:48:42.945599 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:42.945605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:42.945680 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:42.945784 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:42.947204 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.948122 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.949538 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.950069 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.950196 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.953701 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:42.953706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:42.953709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:42.953739 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:42.956627 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.957271 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.957276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.957279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.957433 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:42.957803 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:42.957808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:42.957812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:42.958122 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:42.958309 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:42.958895 (+ 586us) server_negotiation.cc:299] Negotiation successful
0810 18:48:42.959127 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:42.960233 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:42.942798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46779 (local address 127.29.174.194:55449)
0810 18:48:42.943598 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:42.943641 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:42.945350 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:42.945962 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:42.945969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:42.946514 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:42.947045 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.947062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.948239 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.948242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.949404 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:42.949416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:42.950255 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:42.950259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:42.950536 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:42.951180 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:42.951214 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:42.953496 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:42.956760 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.956767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.956777 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.957141 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.957522 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:42.957525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:42.957527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:42.957688 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:42.958230 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:42.958235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:42.958538 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 18:48:42.958765 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 18:48:42.959210 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":305,"threads_started":1}
I20260810 18:48:42.961462 2726 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46779
I20260810 18:48:42.961707 2726 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:42.962332 2726 heartbeater.cc:511] Master 127.29.174.254:46779 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
I20260810 18:48:42.963693 2172 ts_manager.cc:194] Registered new tserver with Master: f3b21be3e13445dd97450b64bee72b1c (127.29.174.194:46211)
I20260810 18:48:42.964465 2172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:55449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:42.985390 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:43.302685 2735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:43.302927 2735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:43.303015 2735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:43.310173 2735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:43.310261 2735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:43.310403 2735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:43.319418 2735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:43.320497 2735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:43.321682 2735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:43.333704 2740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.333730 2741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:43.333967 2745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:43.334520 2735 server_base.cc:1034] running on GCE node
I20260810 18:48:43.335045 2735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:43.335937 2735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:43.337152 2735 hybrid_clock.cc:648] HybridClock initialized: now 1786387723337092 us; error 39 us; skew 500 ppm
I20260810 18:48:43.337437 2735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:43 dist-test-slave-6zbq krb5kdc[2101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:43.340471 2735 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:43.341773 2735 webserver.cc:460] Webserver started at http://127.29.174.195:41065/ using document root <none> and password file <none>
I20260810 18:48:43.342430 2735 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:43.342562 2735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:43.342837 2735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:43.345562 2735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "e6986977f8864c0b8550ad5a99e7caac"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-6zbq"
server_key: "052c318e258d8d374fb2e4b3340bc394"
server_key_iv: "16794603386ad4f9d19651a8a364e348"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.346300 2735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "e6986977f8864c0b8550ad5a99e7caac"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-6zbq"
server_key: "052c318e258d8d374fb2e4b3340bc394"
server_key_iv: "16794603386ad4f9d19651a8a364e348"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.350945 2735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:43.354007 2758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.355183 2735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 18:48:43.355330 2735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "e6986977f8864c0b8550ad5a99e7caac"
format_stamp: "Formatted at 2026-08-10 18:48:43 on dist-test-slave-6zbq"
server_key: "052c318e258d8d374fb2e4b3340bc394"
server_key_iv: "16794603386ad4f9d19651a8a364e348"
server_key_version: "encryptionkey@0"
I20260810 18:48:43.355506 2735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:43.376953 2735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:43.380481 2735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:43.380726 2735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:43.381589 2735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:43.382835 2735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:43.382915 2735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.383028 2735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:43.383080 2735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:43.413552 2735 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:35091
I20260810 18:48:43.413611 2979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:35091 every 8 connection(s)
I20260810 18:48:43.414794 2735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:43.421629 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2735
I20260810 18:48:43.421800 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:43.422200 30395 external_mini_cluster.cc:1428] Setting key 2f061ba40fa7a71d6598ce991e21e9be
Aug 10 18:48:43 dist-test-slave-6zbq krb5kdc[2101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387723, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:43.432992 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.417106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:34705 (local address 127.29.174.254:46779)
0810 18:48:43.417264 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.417270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.418372 (+ 1102us) server_negotiation.cc:407] Connection header received
0810 18:48:43.420128 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.420134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.420215 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.420313 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.421718 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.422563 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.424304 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.424774 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.424877 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.428026 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:43.428031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:43.428035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:43.428067 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:43.430461 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.431172 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.431176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.431178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.431294 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:43.431686 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:43.431689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:43.431692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:43.431951 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:43.432056 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:43.432590 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 18:48:43.432756 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:48:43.433809 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:43.417517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46779 (local address 127.29.174.195:34705)
0810 18:48:43.418197 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:43.418233 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:43.419872 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:43.420539 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:43.420547 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:43.421032 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:43.421551 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.421568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.422704 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.422708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.424172 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:43.424184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:43.424976 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:43.424985 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:43.425195 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:43.425724 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:43.425755 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:43.427842 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:43.430581 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.430587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.430614 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.431071 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.431376 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:43.431379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:43.431381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:43.431605 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:43.432037 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:43.432042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:43.432317 (+ 275us) client_negotiation.cc:769] Sending connection context
0810 18:48:43.432526 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 18:48:43.432869 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":257,"threads_started":1}
I20260810 18:48:43.434931 2980 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46779
I20260810 18:48:43.435257 2980 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:43.435925 2980 heartbeater.cc:511] Master 127.29.174.254:46779 requested a full tablet report, sending...
I20260810 18:48:43.437218 2172 ts_manager.cc:194] Registered new tserver with Master: e6986977f8864c0b8550ad5a99e7caac (127.29.174.195:35091)
I20260810 18:48:43.437983 2172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:34705
I20260810 18:48:43.448592 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:48:43.485754 2493 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:43.483279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.29.174.254:46779)
0810 18:48:43.483482 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:43.483490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:43.483880 (+ 390us) server_negotiation.cc:407] Connection header received
0810 18:48:43.484055 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:43.484061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:43.484130 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:43.484217 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:43.485461 (+ 1244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45196: BlockingRecv error: recv got EOF from 127.0.0.1:45196 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:48:43.488651 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2274
W20260810 18:48:43.491886 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2274: Not authorized: Client connection negotiation failed: client connection to 127.29.174.193:36553: server requires authentication, but client does not have Kerberos credentials available
I20260810 18:48:43.492056 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2274
I20260810 18:48:43.515296 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2500
W20260810 18:48:43.517782 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2500: Not authorized: Client connection negotiation failed: client connection to 127.29.174.194:46211: server requires authentication, but client does not have Kerberos credentials available
I20260810 18:48:43.517917 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2500
I20260810 18:48:43.537505 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2735
W20260810 18:48:43.540283 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2735: Not authorized: Client connection negotiation failed: client connection to 127.29.174.195:35091: server requires authentication, but client does not have Kerberos credentials available
I20260810 18:48:43.540395 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2735
I20260810 18:48:43.559297 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2125
I20260810 18:48:43.560464 2227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:43.660591 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2125
2026-08-10T18:48:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4811 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:43 dist-test-slave-6zbq krb5kdc[3008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:43 dist-test-slave-6zbq krb5kdc[3008](info): set up 2 sockets
Aug 10 18:48:43 dist-test-slave-6zbq krb5kdc[3008](info): commencing operation
krb5kdc: starting...
W20260810 18:48:44.887871 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:45.089741 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41661
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:41661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:45.400264 3026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:45.400509 3026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:45.400624 3026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:45.407791 3026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:45.407898 3026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:45.407964 3026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:45.408041 3026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:45.408139 3026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:45.418897 3026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:41661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:45.420109 3026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:45.421391 3026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:45.432229 3033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:45.432189 3037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:45.432289 3034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:45.432987 3026 server_base.cc:1034] running on GCE node
I20260810 18:48:45.433739 3026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:45.435990 3026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:45.437207 3026 hybrid_clock.cc:648] HybridClock initialized: now 1786387725437191 us; error 114 us; skew 500 ppm
I20260810 18:48:45.437548 3026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387725, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:45.440541 3026 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:45.441956 3026 webserver.cc:460] Webserver started at http://127.29.174.254:35607/ using document root <none> and password file <none>
I20260810 18:48:45.442636 3026 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:45.442751 3026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:45.443056 3026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:45.446058 3026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "6685e902a56c44349fa3cea0219ce201"
format_stamp: "Formatted at 2026-08-10 18:48:45 on dist-test-slave-6zbq"
server_key: "fb76ea9fe49e66a5ca3b714aa945cfa3"
server_key_iv: "5c393caa2a5d2296b89cf1f82ea51315"
server_key_version: "encryptionkey@0"
I20260810 18:48:45.446825 3026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "6685e902a56c44349fa3cea0219ce201"
format_stamp: "Formatted at 2026-08-10 18:48:45 on dist-test-slave-6zbq"
server_key: "fb76ea9fe49e66a5ca3b714aa945cfa3"
server_key_iv: "5c393caa2a5d2296b89cf1f82ea51315"
server_key_version: "encryptionkey@0"
I20260810 18:48:45.451555 3026 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:45.454875 3047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:45.456121 3026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 18:48:45.456295 3026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6685e902a56c44349fa3cea0219ce201"
format_stamp: "Formatted at 2026-08-10 18:48:45 on dist-test-slave-6zbq"
server_key: "fb76ea9fe49e66a5ca3b714aa945cfa3"
server_key_iv: "5c393caa2a5d2296b89cf1f82ea51315"
server_key_version: "encryptionkey@0"
I20260810 18:48:45.456470 3026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:45.468760 3026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:45.472486 3026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:45.472721 3026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:45.489538 3026 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:41661
I20260810 18:48:45.489580 3130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:41661 every 8 connection(s)
I20260810 18:48:45.490825 3026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:45.494318 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:45.496050 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3026
I20260810 18:48:45.496371 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:45.497215 30395 external_mini_cluster.cc:1428] Setting key d15cc0b5ceb44c8fe0115b60836fe589
I20260810 18:48:45.507040 3131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201: Bootstrap starting.
I20260810 18:48:45.509874 3131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201: Neither blocks nor log segments found. Creating new log.
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:45.511143 3131 log.cc:824] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:45.515069 3131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201: No bootstrap required, opened a new log
I20260810 18:48:45.520670 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:45.499056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.29.174.254:41661)
0810 18:48:45.499816 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:45.499827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:45.499863 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:48:45.501402 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:45.501425 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:45.501797 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:45.502105 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:45.503544 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:45.504829 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:45.506068 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:45.506937 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:45.507126 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:45.511768 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:45.511777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:45.511791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:45.511825 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:45.515747 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:45.516817 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:45.516826 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:45.516834 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:45.517049 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:45.517514 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:45.517519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:45.517523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:45.518253 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:45.518590 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:45.518943 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:48:45.519373 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":333,"threads_started":1}
I20260810 18:48:45.524400 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:45.524780 3131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } }
I20260810 18:48:45.525096 3131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:45.525197 3131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6685e902a56c44349fa3cea0219ce201, State: Initialized, Role: FOLLOWER
I20260810 18:48:45.525727 3131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } }
I20260810 18:48:45.525924 3131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:45.526043 3131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:45.526196 3131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:45.528249 3131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } }
I20260810 18:48:45.528683 3131 leader_election.cc:302] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6685e902a56c44349fa3cea0219ce201; no voters:
I20260810 18:48:45.529280 3131 leader_election.cc:288] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:45.529369 3136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:45.531468 3131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:45.533308 3136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [term 1 LEADER]: Becoming Leader. State: Replica: 6685e902a56c44349fa3cea0219ce201, State: Running, Role: LEADER
I20260810 18:48:45.533922 3136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } }
I20260810 18:48:45.536624 3139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6685e902a56c44349fa3cea0219ce201. Latest consensus state: current_term: 1 leader_uuid: "6685e902a56c44349fa3cea0219ce201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } } }
I20260810 18:48:45.536998 3139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:45.537531 3138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6685e902a56c44349fa3cea0219ce201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6685e902a56c44349fa3cea0219ce201" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41661 } } }
I20260810 18:48:45.537781 3138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:45.539778 3153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:45.558774 3153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:45.568105 3153 catalog_manager.cc:1347] Generated new cluster ID: d513952662564afeb44a6da19d03d63e
I20260810 18:48:45.568212 3153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:45.580186 3153 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:45.581712 3153 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:45.583422 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:45.573741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.29.174.254:41661)
0810 18:48:45.573897 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:45.573903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:45.574604 (+ 701us) server_negotiation.cc:407] Connection header received
0810 18:48:45.574757 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:45.574763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:45.574858 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:45.574983 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:45.576049 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:45.577430 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:45.578913 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:45.579505 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:45.579631 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:45.581502 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:45.581510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:45.581515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:45.582540 (+ 1025us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:45.582623 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:45.582660 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:45.582779 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:45.582928 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 18:48:45.583059 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":174,"server-negotiator.queue_time_us":51}
I20260810 18:48:45.597493 3153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6685e902a56c44349fa3cea0219ce201: Generated new TSK 0
I20260810 18:48:45.598280 3153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:45.608877 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3026
I20260810 18:48:45.610066 3122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:45.698050 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3026
2026-08-10T18:48:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2031 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3187](info): set up 2 sockets
Aug 10 18:48:45 dist-test-slave-6zbq krb5kdc[3187](info): commencing operation
krb5kdc: starting...
W20260810 18:48:46.901391 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:47 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:47.093535 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:40349
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:40349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:47.406368 3213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:47.406605 3213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:47.406728 3213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:47.413516 3213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:47.413605 3213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:47.413681 3213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:47.413749 3213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:47.413808 3213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:47.423616 3213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:40349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:40349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:47.424789 3213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:47.425908 3213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:47.436668 3229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:47.436812 3232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:47.436666 3230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:47.437784 3213 server_base.cc:1034] running on GCE node
I20260810 18:48:47.438511 3213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:47.439687 3213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:47.440874 3213 hybrid_clock.cc:648] HybridClock initialized: now 1786387727440825 us; error 43 us; skew 500 ppm
I20260810 18:48:47.441174 3213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:47 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:47.444021 3213 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:47.445338 3213 webserver.cc:460] Webserver started at http://127.29.174.254:45915/ using document root <none> and password file <none>
I20260810 18:48:47.445917 3213 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:47.446030 3213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:47.446296 3213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:47.449029 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "396fc5776d6148ecb19193fe5e36767e"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "ed210dbfe7ddc84a5ed1f36f14191fbb"
server_key_iv: "86b258b1a1cfc3f728d98ba51c7fe70b"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.449707 3213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "396fc5776d6148ecb19193fe5e36767e"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "ed210dbfe7ddc84a5ed1f36f14191fbb"
server_key_iv: "86b258b1a1cfc3f728d98ba51c7fe70b"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.454515 3213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 18:48:47.457499 3242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:47.458660 3213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:47.458824 3213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "396fc5776d6148ecb19193fe5e36767e"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "ed210dbfe7ddc84a5ed1f36f14191fbb"
server_key_iv: "86b258b1a1cfc3f728d98ba51c7fe70b"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.458986 3213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:47.478511 3213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:47.482164 3213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:47.482424 3213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:47.498479 3213 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:40349
I20260810 18:48:47.498497 3334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:40349 every 8 connection(s)
I20260810 18:48:47.499598 3213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:47.502871 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3213
I20260810 18:48:47.502956 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:47.503086 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:47.503911 30395 external_mini_cluster.cc:1428] Setting key c70b2795cdf7e26074fbd9453e333591
I20260810 18:48:47.516064 3337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e: Bootstrap starting.
Aug 10 18:48:47 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:47.519246 3337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:47.520248 3337 log.cc:824] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:47.522869 3337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e: No bootstrap required, opened a new log
I20260810 18:48:47.525775 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:47.505798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.29.174.254:40349)
0810 18:48:47.506528 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:47.506538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:47.506575 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:48:47.508768 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:47.508790 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:47.509126 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:47.509445 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:47.511120 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:47.512322 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:47.514016 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:47.514673 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:47.514803 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:47.519102 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:47.519109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:47.519121 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:47.519162 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:47.521929 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:47.522686 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:47.522692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:47.522697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:47.522852 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:47.523210 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:47.523220 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:47.523222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:47.523673 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:47.523970 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:47.524398 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 18:48:47.524884 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":342,"threads_started":1}
I20260810 18:48:47.530320 3337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } }
I20260810 18:48:47.530676 3337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:47.530761 3337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396fc5776d6148ecb19193fe5e36767e, State: Initialized, Role: FOLLOWER
I20260810 18:48:47.531330 3337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } }
I20260810 18:48:47.531543 3337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:47.531668 3337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:47.531805 3337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:47.533732 3337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } }
I20260810 18:48:47.534247 3337 leader_election.cc:302] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396fc5776d6148ecb19193fe5e36767e; no voters:
I20260810 18:48:47.534773 3337 leader_election.cc:288] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:47.534868 3349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:47.535490 3349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [term 1 LEADER]: Becoming Leader. State: Replica: 396fc5776d6148ecb19193fe5e36767e, State: Running, Role: LEADER
I20260810 18:48:47.536034 3349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } }
I20260810 18:48:47.536598 3337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:47.538805 3351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396fc5776d6148ecb19193fe5e36767e. Latest consensus state: current_term: 1 leader_uuid: "396fc5776d6148ecb19193fe5e36767e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } } }
I20260810 18:48:47.539258 3351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:47.538833 3350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396fc5776d6148ecb19193fe5e36767e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396fc5776d6148ecb19193fe5e36767e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 40349 } } }
I20260810 18:48:47.539641 3350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:47.542763 3357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:47.548108 3357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:47.556139 3357 catalog_manager.cc:1347] Generated new cluster ID: 7b749dab149b40efb3c79b61b56451d3
I20260810 18:48:47.556236 3357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:47.576858 3357 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:47.578051 3357 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:47.584867 3357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396fc5776d6148ecb19193fe5e36767e: Generated new TSK 0
I20260810 18:48:47.585526 3357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:47.635864 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:47.949992 3386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:47.950240 3386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:47.950413 3386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:47.957237 3386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:47.957334 3386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:47.957453 3386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:47.966607 3386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:47.967794 3386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:47.968973 3386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:47.980738 3395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:47.980976 3393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:47.981184 3391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:47.981685 3386 server_base.cc:1034] running on GCE node
I20260810 18:48:47.982277 3386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:47.983114 3386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:47.984309 3386 hybrid_clock.cc:648] HybridClock initialized: now 1786387727984258 us; error 44 us; skew 500 ppm
I20260810 18:48:47.984573 3386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:47 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:47.987454 3386 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:47.988672 3386 webserver.cc:460] Webserver started at http://127.29.174.193:41375/ using document root <none> and password file <none>
I20260810 18:48:47.989250 3386 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:47.989364 3386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:47.989629 3386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:47.992321 3386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "045d4c2264b94dddadb34f6791cbc118"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "60d248da8ac248d27af2e78bd3f07225"
server_key_iv: "012ca9fd3a097fba6b07d4056676636c"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.993023 3386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "045d4c2264b94dddadb34f6791cbc118"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "60d248da8ac248d27af2e78bd3f07225"
server_key_iv: "012ca9fd3a097fba6b07d4056676636c"
server_key_version: "encryptionkey@0"
I20260810 18:48:47.997566 3386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:48.000898 3408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.002135 3386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:48:48.002300 3386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "045d4c2264b94dddadb34f6791cbc118"
format_stamp: "Formatted at 2026-08-10 18:48:47 on dist-test-slave-6zbq"
server_key: "60d248da8ac248d27af2e78bd3f07225"
server_key_iv: "012ca9fd3a097fba6b07d4056676636c"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.002512 3386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.019742 3386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.023532 3386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.023799 3386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.024636 3386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:48.026016 3386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:48.026091 3386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.026219 3386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:48.026271 3386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.056836 3386 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:43063
I20260810 18:48:48.056878 3609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:43063 every 8 connection(s)
I20260810 18:48:48.058049 3386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:48.063448 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3386
I20260810 18:48:48.063632 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:48.064393 30395 external_mini_cluster.cc:1428] Setting key 4af862f0a0e862f850d8cda1f9da580f
Aug 10 18:48:48 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387727, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:48.078651 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.060462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:40849 (local address 127.29.174.254:40349)
0810 18:48:48.060853 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.060858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.061816 (+ 958us) server_negotiation.cc:407] Connection header received
0810 18:48:48.063743 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.063747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.063821 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.063904 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.065764 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.066681 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.068801 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.069179 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.069264 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.072875 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.072881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.072886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.072922 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.075551 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.076420 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.076425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.076427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.076555 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.076999 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.077002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.077004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.077255 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.077366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.078184 (+ 818us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.078383 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":190,"threads_started":1}
I20260810 18:48:48.079785 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.060970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:40349 (local address 127.29.174.193:40849)
0810 18:48:48.061663 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.061699 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.063408 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.064197 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.064209 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.064922 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.065583 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.065604 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.066857 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.066863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.068680 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.068695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.069385 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.069392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.069653 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.070309 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.070360 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.072667 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.075683 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.075692 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.075717 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.076289 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.076673 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.076678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.076682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.076907 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.077371 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.077378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.077751 (+ 373us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.078087 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.078562 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":268,"threads_started":1}
I20260810 18:48:48.081176 3612 heartbeater.cc:347] Connected to a master server at 127.29.174.254:40349
I20260810 18:48:48.081451 3612 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:48.082106 3612 heartbeater.cc:511] Master 127.29.174.254:40349 requested a full tablet report, sending...
I20260810 18:48:48.084146 3268 ts_manager.cc:194] Registered new tserver with Master: 045d4c2264b94dddadb34f6791cbc118 (127.29.174.193:43063)
I20260810 18:48:48.085712 3268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:40849
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:48.111402 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:48.428186 3625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:48.428454 3625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:48.428597 3625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:48.435774 3625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:48.435882 3625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:48.436026 3625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:48.445372 3625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:48.446558 3625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:48.447767 3625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:48.460278 3640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.460309 3641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.460356 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.461057 3625 server_base.cc:1034] running on GCE node
I20260810 18:48:48.461669 3625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:48.462714 3625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:48.463908 3625 hybrid_clock.cc:648] HybridClock initialized: now 1786387728463841 us; error 59 us; skew 500 ppm
I20260810 18:48:48.464200 3625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:48 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:48.467103 3625 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:48.468441 3625 webserver.cc:460] Webserver started at http://127.29.174.194:43077/ using document root <none> and password file <none>
I20260810 18:48:48.469064 3625 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:48.469182 3625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:48.469460 3625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:48.472410 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "64dabe6e73764963b69f7e4e621e5505"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "bfdc30595ab2981f64bc767cf88e262e"
server_key_iv: "781adeb85efd203879c9102028f62ad8"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.473188 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "64dabe6e73764963b69f7e4e621e5505"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "bfdc30595ab2981f64bc767cf88e262e"
server_key_iv: "781adeb85efd203879c9102028f62ad8"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.477921 3625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 18:48:48.481034 3658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.482184 3625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:48.482324 3625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "64dabe6e73764963b69f7e4e621e5505"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "bfdc30595ab2981f64bc767cf88e262e"
server_key_iv: "781adeb85efd203879c9102028f62ad8"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.482538 3625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.509397 3625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.516305 3625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.516588 3625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.517620 3625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:48.519214 3625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:48.519291 3625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.519415 3625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:48.519465 3625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.549933 3625 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:40091
I20260810 18:48:48.549990 3868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:40091 every 8 connection(s)
I20260810 18:48:48.551179 3625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:48.561493 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3625
I20260810 18:48:48.561659 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:48.562306 30395 external_mini_cluster.cc:1428] Setting key 95f61a737098b2354e965c56d2a40c04
Aug 10 18:48:48 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:48.569590 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.553432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:42115 (local address 127.29.174.254:40349)
0810 18:48:48.553638 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:48.553644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:48.554717 (+ 1073us) server_negotiation.cc:407] Connection header received
0810 18:48:48.556413 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:48.556417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:48.556485 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:48.556571 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:48.557932 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.558725 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.560259 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.560664 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.560756 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.564375 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:48.564380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:48.564383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:48.564414 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:48.566834 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.567592 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.567595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.567598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.567732 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:48.568151 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:48.568154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:48.568156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:48.568385 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:48.568495 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:48.569187 (+ 692us) server_negotiation.cc:299] Negotiation successful
0810 18:48:48.569351 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:48:48.570763 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:48.553849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:40349 (local address 127.29.174.194:42115)
0810 18:48:48.554573 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:48.554611 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:48.556158 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:48.556739 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:48.556747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:48.557224 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:48.557756 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.557773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.558865 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.558869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.560139 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:48.560152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:48.560827 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:48.560831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:48.561025 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:48.561731 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:48.561760 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:48.564170 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:48.566959 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.566966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.566979 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.567497 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.567826 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:48.567830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:48.567833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:48.568055 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:48.568501 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:48.568507 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:48.568859 (+ 352us) client_negotiation.cc:769] Sending connection context
0810 18:48:48.569130 (+ 271us) client_negotiation.cc:240] Negotiation successful
0810 18:48:48.569576 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":277,"threads_started":1}
I20260810 18:48:48.572008 3869 heartbeater.cc:347] Connected to a master server at 127.29.174.254:40349
I20260810 18:48:48.572305 3869 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:48.572947 3869 heartbeater.cc:511] Master 127.29.174.254:40349 requested a full tablet report, sending...
I20260810 18:48:48.574293 3268 ts_manager.cc:194] Registered new tserver with Master: 64dabe6e73764963b69f7e4e621e5505 (127.29.174.194:40091)
I20260810 18:48:48.575336 3268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:42115
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:48.609261 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:48.920475 3877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:48.920723 3877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:48.920822 3877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:48.927640 3877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:48.927729 3877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:48.927839 3877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:48.937022 3877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:40349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:48.938177 3877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:48.939432 3877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:48.951507 3893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.951591 3895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:48.951903 3888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:48.952226 3877 server_base.cc:1034] running on GCE node
I20260810 18:48:48.952718 3877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:48.953485 3877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:48.954668 3877 hybrid_clock.cc:648] HybridClock initialized: now 1786387728954618 us; error 42 us; skew 500 ppm
I20260810 18:48:48.954938 3877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:48 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:48.961694 3877 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:48.962967 3877 webserver.cc:460] Webserver started at http://127.29.174.195:38713/ using document root <none> and password file <none>
I20260810 18:48:48.963550 3877 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:48.963660 3877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:48.963929 3877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:48.966688 3877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "4b4817c99ef2403ea7301000ca797114"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "54b11eeeed67f2d0ea942727e48f7298"
server_key_iv: "277b0c0ed86520686993fb1f9d0632c6"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.967415 3877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "4b4817c99ef2403ea7301000ca797114"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "54b11eeeed67f2d0ea942727e48f7298"
server_key_iv: "277b0c0ed86520686993fb1f9d0632c6"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.972003 3877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:48.975154 3902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:48.976330 3877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:48.976468 3877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "4b4817c99ef2403ea7301000ca797114"
format_stamp: "Formatted at 2026-08-10 18:48:48 on dist-test-slave-6zbq"
server_key: "54b11eeeed67f2d0ea942727e48f7298"
server_key_iv: "277b0c0ed86520686993fb1f9d0632c6"
server_key_version: "encryptionkey@0"
I20260810 18:48:48.976639 3877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:48.994309 3877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:48.997668 3877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:48.997910 3877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:48.998828 3877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:49.000021 3877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:49.000095 3877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:49.000221 3877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:49.000277 3877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:49.029969 3877 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37119
I20260810 18:48:49.030010 4100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37119 every 8 connection(s)
I20260810 18:48:49.031230 3877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:49.038084 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3877
I20260810 18:48:49.038230 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:49.038684 30395 external_mini_cluster.cc:1428] Setting key 7e9b34c4c74dd8fac0be0d0dcea558b2
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387728, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:49.049778 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.033668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:46619 (local address 127.29.174.254:40349)
0810 18:48:49.033841 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.033845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.034922 (+ 1077us) server_negotiation.cc:407] Connection header received
0810 18:48:49.036709 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.036715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.036798 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.036887 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.038464 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.039284 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.041163 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.041664 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.041765 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.044882 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.044887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.044890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.044922 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.047385 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.047997 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.048000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.048003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.048113 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.048428 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.048431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.048433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.048674 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.048789 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.049372 (+ 583us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.049535 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:49.050661 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.034054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:40349 (local address 127.29.174.195:46619)
0810 18:48:49.034774 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:49.034816 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:49.036455 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:49.037114 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:49.037122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:49.037660 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:49.038282 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.038307 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.039441 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.039447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.041018 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:49.041031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.041856 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.041861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:49.042047 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.042620 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:49.042646 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:49.044704 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:49.047496 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:49.047502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:49.047512 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:49.047908 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:49.048188 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:49.048191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:49.048193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:49.048350 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:49.048768 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:49.048773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:49.049034 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 18:48:49.049275 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 18:48:49.049646 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":305,"threads_started":1}
I20260810 18:48:49.051798 4101 heartbeater.cc:347] Connected to a master server at 127.29.174.254:40349
I20260810 18:48:49.052086 4101 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:49.052919 4101 heartbeater.cc:511] Master 127.29.174.254:40349 requested a full tablet report, sending...
I20260810 18:48:49.054230 3268 ts_manager.cc:194] Registered new tserver with Master: 4b4817c99ef2403ea7301000ca797114 (127.29.174.195:37119)
I20260810 18:48:49.055001 3268 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:46619
I20260810 18:48:49.065105 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:49.088416 3612 heartbeater.cc:503] Master 127.29.174.254:40349 was elected leader, sending a full tablet report...
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:49.114955 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.102681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.29.174.254:40349)
0810 18:48:49.102849 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.102854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.103330 (+ 476us) server_negotiation.cc:407] Connection header received
0810 18:48:49.103478 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.103482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.103545 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.103616 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.104847 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.105578 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.106838 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.107307 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.107397 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.110761 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.110766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.110769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.110799 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.112964 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.113499 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.113502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.113505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.113624 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.113894 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.113896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.113898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.114178 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.114380 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.114580 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.114731 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:48:49.115497 3268 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54916
I20260810 18:48:49.117450 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3386
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:49.131731 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.118659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55870 (local address 127.29.174.193:43063)
0810 18:48:49.119252 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.119258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.119281 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:48:49.119333 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.119339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.119530 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.119641 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.120643 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.121655 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.122811 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.123343 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.123436 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.126976 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.126987 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.127000 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.127031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.129589 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.130127 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.130132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.130135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.130247 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.130554 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.130557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.130559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.130880 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.131103 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.131277 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.131453 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
W20260810 18:48:49.132833 3605 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55870
W20260810 18:48:49.133796 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3386: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:49.133967 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3386
I20260810 18:48:49.157536 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3625
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:49.171926 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.158309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.29.174.194:40091)
0810 18:48:49.158857 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.158864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.158888 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:48:49.158943 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.158949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.159141 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.159279 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.160481 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.161511 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.162816 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.163356 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.163480 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.167215 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.167225 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.167237 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.167278 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.169662 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.170211 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.170216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.170218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.170327 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.170643 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.170646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.170648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.170957 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.171183 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.171395 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.171583 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":231,"threads_started":1}
W20260810 18:48:49.173247 3860 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34006
W20260810 18:48:49.174273 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3625: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:49.174453 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3625
I20260810 18:48:49.191979 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3877
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[3187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387729, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:49.206779 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:49.192824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.29.174.195:37119)
0810 18:48:49.193440 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:49.193446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:49.193472 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:48:49.193529 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:49.193535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:49.193716 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:49.193832 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:49.195028 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.196059 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.197184 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:49.197706 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:49.197802 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:49.201728 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:49.201740 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:49.201753 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:49.201790 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:49.204491 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.205065 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.205071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.205073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.205191 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:49.205452 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:49.205454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:49.205456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:49.205764 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:49.205984 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:49.206183 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 18:48:49.206500 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":337,"threads_started":1}
W20260810 18:48:49.207943 4096 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59818
W20260810 18:48:49.208927 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3877: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:48:49.209074 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3877
I20260810 18:48:49.233057 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3213
I20260810 18:48:49.234163 3325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:49.335433 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3213
2026-08-10T18:48:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3643 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[4123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[4123](info): set up 2 sockets
Aug 10 18:48:49 dist-test-slave-6zbq krb5kdc[4123](info): commencing operation
krb5kdc: starting...
W20260810 18:48:50.568526 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:50 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:50.777558 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33677
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:33677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:51.089181 4143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:51.089442 4143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:51.089538 4143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:51.096484 4143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:51.096575 4143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:51.096625 4143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:51.096679 4143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:51.096721 4143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:51.105931 4143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:33677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:51.107069 4143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:51.108261 4143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:51.119284 4149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:51.119315 4152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:51.119341 4150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:51.120224 4143 server_base.cc:1034] running on GCE node
I20260810 18:48:51.120952 4143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:51.122260 4143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:51.123451 4143 hybrid_clock.cc:648] HybridClock initialized: now 1786387731123396 us; error 49 us; skew 500 ppm
I20260810 18:48:51.123762 4143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:51 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:51.126597 4143 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:51.127955 4143 webserver.cc:460] Webserver started at http://127.29.174.254:44921/ using document root <none> and password file <none>
I20260810 18:48:51.128588 4143 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:51.128700 4143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:51.128989 4143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:51.131821 4143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "f29389b9760f4faba21cbfce55c57f97"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "a4ee1bf0b842fd004622b64c91e02ce9"
server_key_iv: "1ffd6dd80f56f4d18dd5c70b7ce655a5"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.132562 4143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "f29389b9760f4faba21cbfce55c57f97"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "a4ee1bf0b842fd004622b64c91e02ce9"
server_key_iv: "1ffd6dd80f56f4d18dd5c70b7ce655a5"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.137423 4143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:48:51.140688 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:51.141898 4143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:48:51.142048 4143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "f29389b9760f4faba21cbfce55c57f97"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "a4ee1bf0b842fd004622b64c91e02ce9"
server_key_iv: "1ffd6dd80f56f4d18dd5c70b7ce655a5"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.142232 4143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:51.165165 4143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:51.169154 4143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:51.169395 4143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:51.185885 4143 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:33677
I20260810 18:48:51.185904 4250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:33677 every 8 connection(s)
I20260810 18:48:51.187161 4143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:51.190653 4253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:51.195178 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4143
I20260810 18:48:51.195376 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:51.195971 30395 external_mini_cluster.cc:1428] Setting key 8ec431da9268d72a6c089c66bbca06c3
I20260810 18:48:51.200882 4253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97: Bootstrap starting.
I20260810 18:48:51.204281 4253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:51.205463 4253 log.cc:824] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:51.208635 4253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97: No bootstrap required, opened a new log
Aug 10 18:48:51 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:51.216410 4253 raft_consensus.cc:348] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } }
I20260810 18:48:51.216779 4253 raft_consensus.cc:374] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:51.216889 4253 raft_consensus.cc:729] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29389b9760f4faba21cbfce55c57f97, State: Initialized, Role: FOLLOWER
I20260810 18:48:51.217595 4253 consensus_queue.cc:260] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } }
I20260810 18:48:51.217840 4253 raft_consensus.cc:388] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:51.217976 4253 raft_consensus.cc:482] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:51.218135 4253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:51.218484 4257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:51.197825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.29.174.254:33677)
0810 18:48:51.198714 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:51.198724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:51.198766 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:48:51.200373 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:51.200395 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:51.200738 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:51.201020 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:51.202844 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.204107 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.205798 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.206818 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.206948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:51.211251 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:51.211259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:51.211272 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:51.211305 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:51.214251 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:51.214940 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:51.214946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:51.214952 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:51.215122 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:51.215471 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:51.215475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:51.215480 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:51.216002 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:51.216459 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:51.216936 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 18:48:51.217350 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":406,"threads_started":1}
I20260810 18:48:51.220453 4253 raft_consensus.cc:504] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } }
I20260810 18:48:51.220925 4253 leader_election.cc:302] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f29389b9760f4faba21cbfce55c57f97; no voters:
I20260810 18:48:51.221557 4253 leader_election.cc:288] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:51.221663 4259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:51.222267 4259 raft_consensus.cc:686] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [term 1 LEADER]: Becoming Leader. State: Replica: f29389b9760f4faba21cbfce55c57f97, State: Running, Role: LEADER
I20260810 18:48:51.222903 4259 consensus_queue.cc:237] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } }
I20260810 18:48:51.223678 4253 sys_catalog.cc:564] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:51.225696 4263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f29389b9760f4faba21cbfce55c57f97. Latest consensus state: current_term: 1 leader_uuid: "f29389b9760f4faba21cbfce55c57f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } } }
I20260810 18:48:51.225726 4262 sys_catalog.cc:455] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f29389b9760f4faba21cbfce55c57f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f29389b9760f4faba21cbfce55c57f97" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33677 } } }
I20260810 18:48:51.226241 4262 sys_catalog.cc:458] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:51.227744 4263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:51.227854 4272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:51.232743 4272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:51.241067 4272 catalog_manager.cc:1347] Generated new cluster ID: 679ec0650ede4446958f5f7d9e402a09
I20260810 18:48:51.241176 4272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:51.257530 4272 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:51.259274 4272 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:51.267035 4272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f29389b9760f4faba21cbfce55c57f97: Generated new TSK 0
I20260810 18:48:51.267957 4272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:51.330366 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:51.643751 4289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:51.643996 4289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:51.644109 4289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:51.651098 4289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:51.651189 4289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:51.651330 4289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:51.660567 4289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:51.661654 4289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:51.662830 4289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:51.674556 4302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:51.674571 4297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:51.674623 4299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:51.675649 4289 server_base.cc:1034] running on GCE node
I20260810 18:48:51.676270 4289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:51.677114 4289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:51.678304 4289 hybrid_clock.cc:648] HybridClock initialized: now 1786387731678253 us; error 39 us; skew 500 ppm
I20260810 18:48:51.678612 4289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:51 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:51.681489 4289 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:51.682776 4289 webserver.cc:460] Webserver started at http://127.29.174.193:38609/ using document root <none> and password file <none>
I20260810 18:48:51.683367 4289 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:51.683480 4289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:51.683749 4289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:51.686383 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "518a1856736f4705b10a96a5887ae374"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "007462214e0e93148fbf1627af90c7f5"
server_key_iv: "ed706596c6b4e5625b2ea25d8d4b574b"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.687111 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "518a1856736f4705b10a96a5887ae374"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "007462214e0e93148fbf1627af90c7f5"
server_key_iv: "ed706596c6b4e5625b2ea25d8d4b574b"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.691732 4289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:51.694818 4314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:51.695948 4289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 18:48:51.696094 4289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "518a1856736f4705b10a96a5887ae374"
format_stamp: "Formatted at 2026-08-10 18:48:51 on dist-test-slave-6zbq"
server_key: "007462214e0e93148fbf1627af90c7f5"
server_key_iv: "ed706596c6b4e5625b2ea25d8d4b574b"
server_key_version: "encryptionkey@0"
I20260810 18:48:51.696274 4289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:51.724499 4289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:51.728319 4289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:51.728556 4289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:51.729429 4289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:51.730628 4289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:51.730698 4289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:51.730804 4289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:51.730852 4289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:51.757628 4289 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:44723
I20260810 18:48:51.757683 4503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:44723 every 8 connection(s)
I20260810 18:48:51.758771 4289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:51.759531 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4289
I20260810 18:48:51.759758 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:51.760422 30395 external_mini_cluster.cc:1428] Setting key 2a5e480b6424b93ea5953c0d85baeddf
Aug 10 18:48:51 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387731, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:51.778390 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:51.761050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:33373 (local address 127.29.174.254:33677)
0810 18:48:51.761520 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:51.761526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:51.762562 (+ 1036us) server_negotiation.cc:407] Connection header received
0810 18:48:51.764301 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:51.764306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:51.764380 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:51.764472 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:51.765859 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.766793 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.768628 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.769027 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.769140 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:51.772745 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:51.772752 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:51.772756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:51.772793 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:51.775617 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:51.776263 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:51.776269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:51.776272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:51.776413 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:51.776785 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:51.776790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:51.776793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:51.777115 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:51.777264 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:51.777797 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 18:48:51.778015 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":229,"threads_started":1}
I20260810 18:48:51.779023 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:51.761631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33677 (local address 127.29.174.193:33373)
0810 18:48:51.762416 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:51.762462 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:51.764049 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:51.764707 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:51.764715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:51.765187 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:51.765696 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:51.765714 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.766942 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.766946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:51.768468 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:51.768485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:51.769239 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:51.769247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:51.769464 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:51.770137 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:51.770167 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:51.772528 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:51.775738 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:51.775744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:51.775754 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:51.776137 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:51.776507 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:51.776510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:51.776512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:51.776672 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:51.777223 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:51.777228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:51.777488 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 18:48:51.777687 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:48:51.778016 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":293,"threads_started":1}
I20260810 18:48:51.780395 4504 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33677
I20260810 18:48:51.780649 4504 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:51.781297 4504 heartbeater.cc:511] Master 127.29.174.254:33677 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
I20260810 18:48:51.783209 4183 ts_manager.cc:194] Registered new tserver with Master: 518a1856736f4705b10a96a5887ae374 (127.29.174.193:44723)
I20260810 18:48:51.784569 4183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:33373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:51.809105 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:52.117282 4513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:52.117540 4513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:52.117676 4513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:52.124835 4513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:52.124944 4513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:52.125087 4513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:52.134491 4513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:52.135633 4513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:52.136849 4513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:52.149039 4520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.149083 4521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.149717 4525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:52.150274 4513 server_base.cc:1034] running on GCE node
I20260810 18:48:52.150861 4513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:52.151713 4513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:52.152909 4513 hybrid_clock.cc:648] HybridClock initialized: now 1786387732152863 us; error 39 us; skew 500 ppm
I20260810 18:48:52.153189 4513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:52.156292 4513 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:52.157995 4513 webserver.cc:460] Webserver started at http://127.29.174.194:42717/ using document root <none> and password file <none>
I20260810 18:48:52.158708 4513 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:52.158824 4513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:52.159104 4513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:52.161942 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "c976d79572ce4ddc8427dae41a72d41a"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "86aadb9afed49b57d2cea775342fd69e"
server_key_iv: "47655a1a9e8138985b99287ea2587207"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.162719 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "c976d79572ce4ddc8427dae41a72d41a"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "86aadb9afed49b57d2cea775342fd69e"
server_key_iv: "47655a1a9e8138985b99287ea2587207"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.167716 4513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 18:48:52.170933 4538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.172407 4513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:48:52.172569 4513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "c976d79572ce4ddc8427dae41a72d41a"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "86aadb9afed49b57d2cea775342fd69e"
server_key_iv: "47655a1a9e8138985b99287ea2587207"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.172763 4513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:52.199498 4513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:52.203586 4513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:52.203855 4513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:52.204988 4513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:52.206435 4513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:52.206516 4513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.206635 4513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:52.206696 4513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.238224 4513 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:34123
I20260810 18:48:52.238279 4721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:34123 every 8 connection(s)
I20260810 18:48:52.239488 4513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:52.246420 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4513
I20260810 18:48:52.246613 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:52.247004 30395 external_mini_cluster.cc:1428] Setting key ac80f1b0d4feb17df8e48d5f1e05fcb4
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:52.259483 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.242537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:57537 (local address 127.29.174.254:33677)
0810 18:48:52.242728 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.242733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.243827 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 18:48:52.245923 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.245927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.245998 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.246087 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.247875 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.248656 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.250185 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.250617 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.250729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.253975 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.253980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.253983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.254014 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.256587 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.257238 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.257243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.257247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.257380 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.257775 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.257780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.257784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.258050 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.258196 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.258909 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.259143 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:48:52.260440 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.242945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33677 (local address 127.29.174.194:57537)
0810 18:48:52.243662 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:52.243701 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:52.245665 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:52.246389 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:52.246401 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:52.247038 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:52.247700 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.247729 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.248801 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.248807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.250032 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.250051 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.250837 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.250844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.251070 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.251700 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:52.251724 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:52.253764 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:52.256712 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.256721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.256736 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.257122 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.257480 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.257485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.257489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.257663 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.258174 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:52.258181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:52.258547 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 18:48:52.258799 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 18:48:52.259314 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":289,"threads_started":1}
I20260810 18:48:52.261839 4723 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33677
I20260810 18:48:52.262230 4723 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:52.263049 4723 heartbeater.cc:511] Master 127.29.174.254:33677 requested a full tablet report, sending...
I20260810 18:48:52.264508 4183 ts_manager.cc:194] Registered new tserver with Master: c976d79572ce4ddc8427dae41a72d41a (127.29.174.194:34123)
I20260810 18:48:52.265303 4183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:57537
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:52.299149 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:52.631268 4730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:52.631520 4730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:52.631610 4730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:52.638636 4730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:52.638743 4730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:52.638854 4730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:52.648449 4730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:33677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:52.649560 4730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:52.650800 4730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:52.663403 4738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.663450 4742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:52.663478 4739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:52.664314 4730 server_base.cc:1034] running on GCE node
I20260810 18:48:52.664844 4730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:52.665665 4730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:52.666864 4730 hybrid_clock.cc:648] HybridClock initialized: now 1786387732666823 us; error 32 us; skew 500 ppm
I20260810 18:48:52.667244 4730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:52.670447 4730 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:52.671840 4730 webserver.cc:460] Webserver started at http://127.29.174.195:43211/ using document root <none> and password file <none>
I20260810 18:48:52.672474 4730 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:52.672581 4730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:52.672857 4730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:52.675726 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "03b89da6dc0d42368e50efa178f665aa"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "4a8959c6f5f18e98563cf187dc06eea8"
server_key_iv: "b5c0cfd0a5522e0c953d82e5b669eeb8"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.676486 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "03b89da6dc0d42368e50efa178f665aa"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "4a8959c6f5f18e98563cf187dc06eea8"
server_key_iv: "b5c0cfd0a5522e0c953d82e5b669eeb8"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.681813 4730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 18:48:52.685038 4752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.686254 4730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:52.686431 4730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "03b89da6dc0d42368e50efa178f665aa"
format_stamp: "Formatted at 2026-08-10 18:48:52 on dist-test-slave-6zbq"
server_key: "4a8959c6f5f18e98563cf187dc06eea8"
server_key_iv: "b5c0cfd0a5522e0c953d82e5b669eeb8"
server_key_version: "encryptionkey@0"
I20260810 18:48:52.686612 4730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:52.701711 4730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:52.705199 4730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:52.705446 4730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:52.706292 4730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:52.707518 4730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:52.707594 4730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.707732 4730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:52.707789 4730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:52.738847 4730 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34609
I20260810 18:48:52.738910 4957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34609 every 8 connection(s)
I20260810 18:48:52.740109 4730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:52.749790 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4730
I20260810 18:48:52.749991 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:52.750530 30395 external_mini_cluster.cc:1428] Setting key 60a373ecdfdba4b27c16dbadf62cc482
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:52.760041 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.742494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:43761 (local address 127.29.174.254:33677)
0810 18:48:52.742658 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.742663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.743772 (+ 1109us) server_negotiation.cc:407] Connection header received
0810 18:48:52.745651 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.745656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.745745 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.745847 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.747610 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.748337 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.749803 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.750421 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.750543 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.754667 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.754673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.754676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.754709 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.757049 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.757811 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.757815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.757817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.757933 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.758370 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.758375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.758379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.758686 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.758823 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.759473 (+ 650us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.759684 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:48:52.760975 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.742927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:33677 (local address 127.29.174.195:43761)
0810 18:48:52.743617 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:52.743654 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:52.745349 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:52.746110 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:52.746121 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:52.746866 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:52.747438 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.747456 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.748460 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.748464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.749656 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:52.749669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.750638 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.750646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:52.750857 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.751551 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:52.751574 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:52.754451 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:52.757182 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.757191 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.757207 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.757685 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.758035 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:52.758041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:52.758045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:52.758230 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:52.758821 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:52.758829 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:52.759128 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 18:48:52.759352 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 18:48:52.759826 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":270,"threads_started":1}
I20260810 18:48:52.762555 4959 heartbeater.cc:347] Connected to a master server at 127.29.174.254:33677
I20260810 18:48:52.762830 4959 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:52.763489 4959 heartbeater.cc:511] Master 127.29.174.254:33677 requested a full tablet report, sending...
I20260810 18:48:52.764761 4183 ts_manager.cc:194] Registered new tserver with Master: 03b89da6dc0d42368e50efa178f665aa (127.29.174.195:34609)
I20260810 18:48:52.765645 4183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:43761
I20260810 18:48:52.766490 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:52.787727 4504 heartbeater.cc:503] Master 127.29.174.254:33677 was elected leader, sending a full tablet report...
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:52.818395 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.804037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.29.174.193:44723)
0810 18:48:52.804644 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.804651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.804675 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:48:52.804741 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.804747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.804960 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.805102 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.806515 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.807511 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.808724 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.809265 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.809376 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.813098 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.813109 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.813128 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.813166 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.815770 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.816371 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.816377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.816379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.816506 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.816861 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.816864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.816866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.817190 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.817476 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.817774 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.818025 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":275,"threads_started":1}
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:52.835112 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.822288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.29.174.254:33677)
0810 18:48:52.822496 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.822502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.822645 (+ 143us) server_negotiation.cc:407] Connection header received
0810 18:48:52.822796 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.822802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.822867 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.822946 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.824114 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.824843 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.826141 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.826647 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.826765 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.830404 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.830412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.830417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.830457 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.832632 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.833204 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.833210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.833214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.833344 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.833748 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.833754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.833757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.834146 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.834429 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.834668 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.834835 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 18:48:52.835608 4182 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41602
I20260810 18:48:52.836541 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4289
I20260810 18:48:52.848289 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.837420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.29.174.193:44723)
0810 18:48:52.837602 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.837607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.837753 (+ 146us) server_negotiation.cc:407] Connection header received
0810 18:48:52.837897 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.837903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.837992 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.838071 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.839237 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.839891 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.841037 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.841468 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.841578 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.843639 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.843644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.843647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.843676 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.845958 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.846587 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.846592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.846596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.846724 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.847053 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.847059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.847062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.847294 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.847504 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.847807 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.848008 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:52.848909 4497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:52.928387 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4289
I20260810 18:48:52.949398 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4513
Aug 10 18:48:52 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:52.964130 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:52.950144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.29.174.194:34123)
0810 18:48:52.950646 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:52.950653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:52.950680 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:48:52.950740 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:52.950746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:52.950951 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:52.951078 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:52.952177 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.953131 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.954600 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:52.955166 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:52.955284 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:52.959138 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:52.959149 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:52.959163 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:52.959204 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:52.961861 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.962496 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.962503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.962505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.962629 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:52.962915 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:52.962918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:52.962920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:52.963232 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:52.963464 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:52.963652 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:48:52.963843 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":254,"threads_started":1}
I20260810 18:48:52.965298 4715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:53.046526 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4513
I20260810 18:48:53.067348 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4730
Aug 10 18:48:53 dist-test-slave-6zbq krb5kdc[4123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:53.083532 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:53.068299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.29.174.195:34609)
0810 18:48:53.068960 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:53.068967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:53.068997 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:48:53.069056 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:53.069062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:53.069264 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:53.069391 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:53.070550 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.071585 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.072754 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:53.073285 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:53.073380 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:53.077934 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:53.077946 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:53.077962 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:53.077999 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:53.080972 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.081708 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.081715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.081717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.081846 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:53.082121 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:53.082124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:53.082126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:53.082512 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:53.082742 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:53.082971 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 18:48:53.083165 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":362,"threads_started":1}
I20260810 18:48:53.084785 4951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:53.165335 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4730
I20260810 18:48:53.184707 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4143
I20260810 18:48:53.185789 4244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:53.286595 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4143
2026-08-10T18:48:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3948 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:48:53 dist-test-slave-6zbq krb5kdc[4993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:48:53 dist-test-slave-6zbq krb5kdc[4993](info): set up 2 sockets
Aug 10 18:48:53 dist-test-slave-6zbq krb5kdc[4993](info): commencing operation
krb5kdc: starting...
W20260810 18:48:54.506961 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:48:54 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:48:54.703652 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36155
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:55.020079 5010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:55.020359 5010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:55.020493 5010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:55.027442 5010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:55.027544 5010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:55.027619 5010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:55.027709 5010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:55.027774 5010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:55.037180 5010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:55.038388 5010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:55.039613 5010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:55.050555 5019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:55.050640 5025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:55.050587 5018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:55.051303 5010 server_base.cc:1034] running on GCE node
I20260810 18:48:55.051997 5010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:55.053251 5010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:55.054445 5010 hybrid_clock.cc:648] HybridClock initialized: now 1786387735054397 us; error 40 us; skew 500 ppm
I20260810 18:48:55.054766 5010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:55 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387735, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:55.062052 5010 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:48:55.063234 5010 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:55.063337 5010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:55.063647 5010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:55.066510 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "f0e10f9064d5472bbec2850532f3777a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "ead5a25cf37c5c39ebc50442889d7892"
server_key_iv: "d5a3d9b3358af303b785a62540c9fe77"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.067289 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "f0e10f9064d5472bbec2850532f3777a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "ead5a25cf37c5c39ebc50442889d7892"
server_key_iv: "d5a3d9b3358af303b785a62540c9fe77"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.071935 5010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:55.075179 5035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:55.076378 5010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:55.076534 5010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "f0e10f9064d5472bbec2850532f3777a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "ead5a25cf37c5c39ebc50442889d7892"
server_key_iv: "d5a3d9b3358af303b785a62540c9fe77"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.076732 5010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:55.090507 5010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:55.094383 5010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:55.094623 5010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:55.111209 5010 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:36155
I20260810 18:48:55.111229 5120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:36155 every 8 connection(s)
I20260810 18:48:55.112015 5010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:55.115417 5121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:55.120378 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 5010
I20260810 18:48:55.120605 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:55.121384 30395 external_mini_cluster.cc:1428] Setting key c0ff8876d9567613c1ef2e68a2b752b8
I20260810 18:48:55.126533 5121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: Bootstrap starting.
I20260810 18:48:55.129899 5121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:55.130885 5121 log.cc:824] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:55.133332 5121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: No bootstrap required, opened a new log
Aug 10 18:48:55 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:55.140853 5121 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } }
I20260810 18:48:55.141212 5121 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:55.141309 5121 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e10f9064d5472bbec2850532f3777a, State: Initialized, Role: FOLLOWER
I20260810 18:48:55.141844 5121 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } }
I20260810 18:48:55.142066 5121 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:55.142199 5121 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:55.142385 5121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:55.143572 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.123479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.29.174.254:36155)
0810 18:48:55.124283 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:55.124294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:55.124337 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:48:55.125989 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:55.126010 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:55.126405 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:55.126737 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:55.128597 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.129945 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.131299 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.131959 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.132124 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.136139 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:55.136147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:55.136163 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:55.136219 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:55.139216 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:55.139868 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:55.139877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:55.139883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:55.140053 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:55.140421 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:55.140424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:55.140426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:55.140957 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:55.141250 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:55.142221 (+ 971us) server_negotiation.cc:299] Negotiation successful
0810 18:48:55.142608 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":374,"threads_started":1}
I20260810 18:48:55.145107 5121 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } }
I20260810 18:48:55.145565 5121 leader_election.cc:302] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0e10f9064d5472bbec2850532f3777a; no voters:
I20260810 18:48:55.146178 5121 leader_election.cc:288] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:55.146313 5126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:55.146885 5126 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [term 1 LEADER]: Becoming Leader. State: Replica: f0e10f9064d5472bbec2850532f3777a, State: Running, Role: LEADER
I20260810 18:48:55.147459 5126 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } }
I20260810 18:48:55.148875 5121 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:55.150133 5129 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0e10f9064d5472bbec2850532f3777a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } } }
I20260810 18:48:55.150518 5129 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:55.151242 5127 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0e10f9064d5472bbec2850532f3777a. Latest consensus state: current_term: 1 leader_uuid: "f0e10f9064d5472bbec2850532f3777a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e10f9064d5472bbec2850532f3777a" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36155 } } }
I20260810 18:48:55.151630 5127 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:55.152715 5134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:55.157708 5134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:55.165959 5134 catalog_manager.cc:1347] Generated new cluster ID: 395d163fad984e9596026a62963e6f77
I20260810 18:48:55.166066 5134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:55.188149 5134 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:55.189416 5134 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:55.201658 5134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: Generated new TSK 0
I20260810 18:48:55.202417 5134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:48:55.255956 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:55.571343 5165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:55.571612 5165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:55.571733 5165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:55.578768 5165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:55.578863 5165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:55.579015 5165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:55.588356 5165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:55.589540 5165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:55.590764 5165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:55.603166 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:55.603153 5181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:55.603407 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:55.604249 5165 server_base.cc:1034] running on GCE node
I20260810 18:48:55.604792 5165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:55.605646 5165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:55.606837 5165 hybrid_clock.cc:648] HybridClock initialized: now 1786387735606794 us; error 37 us; skew 500 ppm
I20260810 18:48:55.607122 5165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:55 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387735, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:55.610008 5165 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:48:55.611093 5165 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:55.611207 5165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:55.611493 5165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:55.614192 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "72d7366c4f5a42b5a7d1658667ab843a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "de7c75e85608ce74f1dcbf4426a6cb38"
server_key_iv: "29ca52ece4456df64f7abbea4f4e1ec3"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.614984 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "72d7366c4f5a42b5a7d1658667ab843a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "de7c75e85608ce74f1dcbf4426a6cb38"
server_key_iv: "29ca52ece4456df64f7abbea4f4e1ec3"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.619565 5165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:55.622720 5189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:55.623864 5165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:48:55.624006 5165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "72d7366c4f5a42b5a7d1658667ab843a"
format_stamp: "Formatted at 2026-08-10 18:48:55 on dist-test-slave-6zbq"
server_key: "de7c75e85608ce74f1dcbf4426a6cb38"
server_key_iv: "29ca52ece4456df64f7abbea4f4e1ec3"
server_key_version: "encryptionkey@0"
I20260810 18:48:55.624203 5165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:55.635380 5165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:55.639213 5165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:55.639449 5165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:55.640295 5165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:55.641496 5165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:55.641587 5165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:55.641685 5165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:55.641752 5165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:55.671315 5165 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:40445
I20260810 18:48:55.671373 5401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:40445 every 8 connection(s)
I20260810 18:48:55.672070 5165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:55.674760 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 5165
I20260810 18:48:55.674957 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:55.675757 30395 external_mini_cluster.cc:1428] Setting key f4565fc27c22e45edbf6956e0c8ce112
Aug 10 18:48:55 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387735, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:55.692430 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.674192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:53891 (local address 127.29.174.254:36155)
0810 18:48:55.674658 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:55.674664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:55.675672 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 18:48:55.677734 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:55.677739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:55.677831 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:55.677922 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:55.679794 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.680522 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.682526 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.682910 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.683002 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.686649 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:55.686654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:55.686657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:55.686697 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:55.689595 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:55.690277 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:55.690282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:55.690285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:55.690451 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:55.690834 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:55.690838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:55.690841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:55.691169 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:55.691307 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:55.691899 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 18:48:55.692104 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":243,"threads_started":1}
I20260810 18:48:55.693050 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:55.674698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36155 (local address 127.29.174.193:53891)
0810 18:48:55.675485 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:55.675542 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:55.677503 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:55.678173 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:55.678183 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:55.678882 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:55.679613 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.679638 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.680684 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.680689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.682402 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:55.682417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:55.683113 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:55.683119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:55.683364 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:55.683979 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:55.684006 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:55.686421 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:55.689744 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:55.689750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:55.689761 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:55.690136 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:55.690542 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:55.690545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:55.690547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:55.690727 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:55.691266 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:55.691271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:55.691532 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 18:48:55.691752 (+ 220us) client_negotiation.cc:240] Negotiation successful
0810 18:48:55.692076 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":302,"threads_started":1}
I20260810 18:48:55.694471 5402 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36155
I20260810 18:48:55.694835 5402 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:55.695523 5402 heartbeater.cc:511] Master 127.29.174.254:36155 requested a full tablet report, sending...
I20260810 18:48:55.697705 5064 ts_manager.cc:194] Registered new tserver with Master: 72d7366c4f5a42b5a7d1658667ab843a (127.29.174.193:40445)
I20260810 18:48:55.699203 5064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:53891
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:48:55.725241 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:56.047711 5417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:56.047992 5417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:56.048089 5417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:56.055394 5417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:56.055497 5417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:56.055619 5417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:56.065162 5417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:56.066294 5417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:56.067533 5417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:56.079742 5427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:56.079845 5426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:56.080214 5430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:56.080583 5417 server_base.cc:1034] running on GCE node
I20260810 18:48:56.081123 5417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:56.081964 5417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:56.083148 5417 hybrid_clock.cc:648] HybridClock initialized: now 1786387736083097 us; error 41 us; skew 500 ppm
I20260810 18:48:56.083415 5417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:56 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387736, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:56.086272 5417 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:48:56.087297 5417 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:56.087386 5417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:56.087648 5417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:56.090319 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "5db142a8d53245e2a8df5d5179eb4c4f"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "41ff863551f155da1f11038f35019419"
server_key_iv: "c206ee2b21ae3a733138c12580018efc"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.091111 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "5db142a8d53245e2a8df5d5179eb4c4f"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "41ff863551f155da1f11038f35019419"
server_key_iv: "c206ee2b21ae3a733138c12580018efc"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.095738 5417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:48:56.098901 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.100059 5417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:56.100203 5417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "5db142a8d53245e2a8df5d5179eb4c4f"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "41ff863551f155da1f11038f35019419"
server_key_iv: "c206ee2b21ae3a733138c12580018efc"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.100379 5417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:56.116019 5417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:56.120616 5417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:56.120873 5417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:56.121869 5417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:56.124013 5417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:56.124154 5417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.124246 5417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:56.124325 5417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.157368 5417 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:44859
I20260810 18:48:56.157413 5626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:44859 every 8 connection(s)
I20260810 18:48:56.158303 5417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:56.163723 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 5417
I20260810 18:48:56.163919 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:56.164427 30395 external_mini_cluster.cc:1428] Setting key 6bd5ac1f7bdb7ff0353b29a51f2bbe33
Aug 10 18:48:56 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387736, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:56.179440 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.161894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:37453 (local address 127.29.174.254:36155)
0810 18:48:56.162147 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.162154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.162553 (+ 399us) server_negotiation.cc:407] Connection header received
0810 18:48:56.164749 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.164753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.164826 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.164912 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.166746 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.167473 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.169643 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.170045 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.170145 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.173938 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:56.173943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:56.173946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:56.173978 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:56.176421 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.177230 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.177235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.177238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.177391 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.177758 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.177763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.177767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.178079 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:56.178223 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.178868 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.179121 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 18:48:56.180086 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.161529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36155 (local address 127.29.174.194:37453)
0810 18:48:56.162336 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.162410 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.164456 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.165153 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.165161 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.165784 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:56.166540 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.166562 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.167655 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.167660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.169488 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.169503 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.170264 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.170270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.170550 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.171208 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:56.171239 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:56.173733 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:56.176551 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:56.176560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:56.176576 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:56.177107 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:56.177487 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:56.177490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:56.177493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:56.177661 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:56.178180 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:56.178185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:56.178497 (+ 312us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.178713 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.179051 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":303,"threads_started":1}
I20260810 18:48:56.181473 5629 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36155
I20260810 18:48:56.181734 5629 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:56.182334 5629 heartbeater.cc:511] Master 127.29.174.254:36155 requested a full tablet report, sending...
I20260810 18:48:56.183667 5064 ts_manager.cc:194] Registered new tserver with Master: 5db142a8d53245e2a8df5d5179eb4c4f (127.29.174.194:44859)
I20260810 18:48:56.184437 5064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:37453
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:48:56.217046 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:48:56.536561 5636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:56.536867 5636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:56.537014 5636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:56.544581 5636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:56.544689 5636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:56.544836 5636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:56.554466 5636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:56.555621 5636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:56.556809 5636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:56.568805 5643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:56.568929 5642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:56.568893 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:56.569784 5636 server_base.cc:1034] running on GCE node
I20260810 18:48:56.570407 5636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:56.571269 5636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:56.572459 5636 hybrid_clock.cc:648] HybridClock initialized: now 1786387736572414 us; error 39 us; skew 500 ppm
I20260810 18:48:56.572736 5636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:56 dist-test-slave-6zbq krb5kdc[4993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387736, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:56.575563 5636 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:48:56.576577 5636 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:56.576675 5636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:56.576951 5636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:56.579901 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "eaece6422912447a90b46a3ff4c7f012"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "5b8413294e980c56edf421a512d41af1"
server_key_iv: "785f7b94d3b3a0d68075cf695fe5e19f"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.580745 5636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "eaece6422912447a90b46a3ff4c7f012"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "5b8413294e980c56edf421a512d41af1"
server_key_iv: "785f7b94d3b3a0d68075cf695fe5e19f"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.585408 5636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 18:48:56.588485 5659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.589582 5636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:48:56.589747 5636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "eaece6422912447a90b46a3ff4c7f012"
format_stamp: "Formatted at 2026-08-10 18:48:56 on dist-test-slave-6zbq"
server_key: "5b8413294e980c56edf421a512d41af1"
server_key_iv: "785f7b94d3b3a0d68075cf695fe5e19f"
server_key_version: "encryptionkey@0"
I20260810 18:48:56.589931 5636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:56.606282 5636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:56.611148 5636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:56.611419 5636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:56.612314 5636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:56.613555 5636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:56.613660 5636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.613765 5636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:56.613837 5636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:56.644676 5636 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:39869
I20260810 18:48:56.644734 5844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:39869 every 8 connection(s)
I20260810 18:48:56.645560 5636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:56.655951 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 5636
I20260810 18:48:56.656332 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:56.656941 30395 external_mini_cluster.cc:1428] Setting key 71ae390364b2267cc7de0b8f38fe30db
Aug 10 18:48:56 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387736, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:48:56.664686 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.648160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:52113 (local address 127.29.174.254:36155)
0810 18:48:56.648363 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.648368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.649401 (+ 1033us) server_negotiation.cc:407] Connection header received
0810 18:48:56.651207 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.651213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.651293 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.651378 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.652782 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.653584 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.655146 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.655554 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.655663 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.659315 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:56.659320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:56.659323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:56.659357 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:56.661829 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.662541 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.662545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.662548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.662659 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.663051 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.663054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.663056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.663379 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:56.663518 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.664155 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.664365 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:48:56.665606 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.648541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36155 (local address 127.29.174.195:52113)
0810 18:48:56.649260 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.649295 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.650958 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.651612 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.651620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.652100 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:56.652626 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.652642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.653728 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.653731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.655006 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.655020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.655773 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.655780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.656001 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.656767 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:56.656797 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:56.659104 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:56.661956 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:56.661965 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:56.661980 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:56.662423 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:56.662760 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:56.662766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:56.662769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:56.662939 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:56.663510 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:56.663518 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:56.663824 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.664047 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.664531 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":271,"threads_started":1}
I20260810 18:48:56.666872 5846 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36155
I20260810 18:48:56.667138 5846 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:56.667748 5846 heartbeater.cc:511] Master 127.29.174.254:36155 requested a full tablet report, sending...
I20260810 18:48:56.669008 5064 ts_manager.cc:194] Registered new tserver with Master: eaece6422912447a90b46a3ff4c7f012 (127.29.174.195:39869)
I20260810 18:48:56.669734 5064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:52113
I20260810 18:48:56.672130 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:56.689533 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.677504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.29.174.254:36155)
0810 18:48:56.677647 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.677650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.678156 (+ 506us) server_negotiation.cc:407] Connection header received
0810 18:48:56.678315 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.678320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.678430 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.678506 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.679878 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.680590 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.681841 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.682293 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.682428 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.684555 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:56.684560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:56.684563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:56.684592 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:56.686806 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.687375 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.687381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.687385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.687509 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.687861 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.687866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.687870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.688282 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:56.688539 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.688924 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.689216 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:48:56.693244 5064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:56.699872 5064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:56.701915 5402 heartbeater.cc:503] Master 127.29.174.254:36155 was elected leader, sending a full tablet report...
I20260810 18:48:56.733410 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.712502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:40445 (local address 127.0.0.1:33458)
0810 18:48:56.725414 (+ 12912us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.725436 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.725550 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.726284 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.726289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.726315 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:56.726645 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.726657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.728452 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.728457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.730247 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.730260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.732508 (+ 2248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.732515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.732694 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.732727 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.732826 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.732956 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12782,"mutex_wait_us":8,"thread_start_us":12602,"threads_started":1}
I20260810 18:48:56.733566 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.713065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.29.174.193:40445)
0810 18:48:56.724956 (+ 11891us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.724964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.725680 (+ 716us) server_negotiation.cc:407] Connection header received
0810 18:48:56.725781 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.725788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.725995 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.726190 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:56.726806 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.728290 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.730412 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.731790 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.731989 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.732334 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.733097 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.733264 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11708,"thread_start_us":11548,"threads_started":1}
I20260810 18:48:56.733750 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.712300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.29.174.194:44859)
0810 18:48:56.723287 (+ 10987us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.723296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.724316 (+ 1020us) server_negotiation.cc:407] Connection header received
0810 18:48:56.724424 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.724432 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.724701 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.724889 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:56.725819 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.727314 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.730723 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.731656 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.733043 (+ 1387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.733208 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.733284 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.733447 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10830,"thread_start_us":10698,"threads_started":1}
I20260810 18:48:56.734088 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.712721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.29.174.195:39869)
0810 18:48:56.722851 (+ 10130us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.722861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.724881 (+ 2020us) server_negotiation.cc:407] Connection header received
0810 18:48:56.724987 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.724994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.725206 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.725395 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:56.726400 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.727972 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.730706 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.732070 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.732416 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.732572 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.733593 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.733747 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9950,"thread_start_us":9616,"threads_started":1}
I20260810 18:48:56.733469 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.712107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:39869 (local address 127.0.0.1:57450)
0810 18:48:56.724588 (+ 12481us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.724607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.724753 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.725676 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.725680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.725704 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:56.726137 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.726152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.728150 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.728155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.730562 (+ 2407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.730572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.732715 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.732721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.732873 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.732902 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.732979 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.733082 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12337,"mutex_wait_us":164,"thread_start_us":11330,"threads_started":1}
I20260810 18:48:56.732983 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.711887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:44859 (local address 127.0.0.1:52926)
0810 18:48:56.723851 (+ 11964us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.723906 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.724109 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.725173 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.725185 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.725231 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:56.725646 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.725659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.727437 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.727442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.730535 (+ 3093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.730555 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.731860 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.731867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.732055 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.732202 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.732341 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.732538 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11707,"mutex_wait_us":94,"thread_start_us":12432,"threads_started":1}
I20260810 18:48:56.737437 5729 tablet_service.cc:1467] Processing CreateTablet for tablet ed46a50fa9c543999c01dc194893f72f (DEFAULT_TABLE table=test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:56.737358 5286 tablet_service.cc:1467] Processing CreateTablet for tablet ed46a50fa9c543999c01dc194893f72f (DEFAULT_TABLE table=test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:56.738739 5729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed46a50fa9c543999c01dc194893f72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:56.738862 5286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed46a50fa9c543999c01dc194893f72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:56.739888 5526 tablet_service.cc:1467] Processing CreateTablet for tablet ed46a50fa9c543999c01dc194893f72f (DEFAULT_TABLE table=test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:56.741281 5526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed46a50fa9c543999c01dc194893f72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:56.749770 5868 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Bootstrap starting.
I20260810 18:48:56.753067 5868 tablet_bootstrap.cc:654] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:56.753293 5869 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Bootstrap starting.
I20260810 18:48:56.754482 5868 log.cc:824] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:56.756776 5869 tablet_bootstrap.cc:654] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:56.758060 5869 log.cc:824] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:56.759558 5872 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Bootstrap starting.
I20260810 18:48:56.762439 5872 tablet_bootstrap.cc:654] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:56.763432 5872 log.cc:824] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:56.763953 5869 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: No bootstrap required, opened a new log
I20260810 18:48:56.763990 5868 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: No bootstrap required, opened a new log
I20260810 18:48:56.764249 5869 ts_tablet_manager.cc:1397] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260810 18:48:56.764238 5868 ts_tablet_manager.cc:1397] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20260810 18:48:56.768718 5872 tablet_bootstrap.cc:492] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: No bootstrap required, opened a new log
I20260810 18:48:56.768965 5872 ts_tablet_manager.cc:1397] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 18:48:56.774451 5869 raft_consensus.cc:348] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.774437 5868 raft_consensus.cc:348] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.774853 5869 raft_consensus.cc:374] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:56.774853 5868 raft_consensus.cc:374] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:56.774955 5869 raft_consensus.cc:729] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d7366c4f5a42b5a7d1658667ab843a, State: Initialized, Role: FOLLOWER
I20260810 18:48:56.774955 5868 raft_consensus.cc:729] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaece6422912447a90b46a3ff4c7f012, State: Initialized, Role: FOLLOWER
I20260810 18:48:56.775534 5869 consensus_queue.cc:260] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.775534 5868 consensus_queue.cc:260] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.775764 5872 raft_consensus.cc:348] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.776216 5872 raft_consensus.cc:374] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:56.776356 5872 raft_consensus.cc:729] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db142a8d53245e2a8df5d5179eb4c4f, State: Initialized, Role: FOLLOWER
I20260810 18:48:56.777148 5846 heartbeater.cc:503] Master 127.29.174.254:36155 was elected leader, sending a full tablet report...
I20260810 18:48:56.777040 5872 consensus_queue.cc:260] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.777235 5868 ts_tablet_manager.cc:1428] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 18:48:56.777796 5869 ts_tablet_manager.cc:1428] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 18:48:56.778663 5629 heartbeater.cc:503] Master 127.29.174.254:36155 was elected leader, sending a full tablet report...
I20260810 18:48:56.779215 5872 ts_tablet_manager.cc:1428] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260810 18:48:56.837071 5877 raft_consensus.cc:482] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:56.837352 5877 raft_consensus.cc:504] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.838929 5877 leader_election.cc:288] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db142a8d53245e2a8df5d5179eb4c4f (127.29.174.194:44859), eaece6422912447a90b46a3ff4c7f012 (127.29.174.195:39869)
I20260810 18:48:56.846997 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.839381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:54333 (local address 127.29.174.195:39869)
0810 18:48:56.839779 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.839786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.840255 (+ 469us) server_negotiation.cc:407] Connection header received
0810 18:48:56.840385 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.840390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.840461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.840541 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:56.841215 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.842224 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.844777 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.845950 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.846098 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.846164 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.846624 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.846757 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254}
I20260810 18:48:56.846997 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.839268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49957 (local address 127.29.174.194:44859)
0810 18:48:56.839605 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.839612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.840040 (+ 428us) server_negotiation.cc:407] Connection header received
0810 18:48:56.840246 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.840252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.840331 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.840427 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:48:56.841095 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.842175 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.844770 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.845950 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.846092 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.846164 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.846628 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.846748 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260810 18:48:56.847266 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.839154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:44859 (local address 127.29.174.193:49957)
0810 18:48:56.839914 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.839940 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.840102 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.840449 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.840452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.840473 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:56.840942 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.840957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.842311 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.842317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.844627 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.844643 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.846180 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.846187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.846436 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.846506 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.846609 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.846792 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":92,"thread_start_us":376,"threads_started":1}
I20260810 18:48:56.847286 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.839405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:39869 (local address 127.29.174.193:54333)
0810 18:48:56.840162 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:56.840178 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:56.840279 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:56.840578 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:56.840582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:56.840614 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:48:56.841044 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.841055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.842367 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.842372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.844632 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:56.844643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.846214 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.846220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:56.846431 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.846505 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 18:48:56.846602 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:48:56.846795 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":155,"thread_start_us":431,"threads_started":1}
I20260810 18:48:56.848270 5555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed46a50fa9c543999c01dc194893f72f" candidate_uuid: "72d7366c4f5a42b5a7d1658667ab843a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" is_pre_election: true
I20260810 18:48:56.848268 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed46a50fa9c543999c01dc194893f72f" candidate_uuid: "72d7366c4f5a42b5a7d1658667ab843a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaece6422912447a90b46a3ff4c7f012" is_pre_election: true
I20260810 18:48:56.848773 5555 raft_consensus.cc:2454] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72d7366c4f5a42b5a7d1658667ab843a in term 0.
I20260810 18:48:56.848795 5763 raft_consensus.cc:2454] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72d7366c4f5a42b5a7d1658667ab843a in term 0.
I20260810 18:48:56.849452 5192 leader_election.cc:302] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d7366c4f5a42b5a7d1658667ab843a, eaece6422912447a90b46a3ff4c7f012; no voters:
I20260810 18:48:56.849852 5877 raft_consensus.cc:2781] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:56.850010 5877 raft_consensus.cc:482] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:56.850149 5877 raft_consensus.cc:3037] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:56.852213 5877 raft_consensus.cc:504] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.852725 5877 leader_election.cc:288] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [CANDIDATE]: Term 1 election: Requested vote from peers 5db142a8d53245e2a8df5d5179eb4c4f (127.29.174.194:44859), eaece6422912447a90b46a3ff4c7f012 (127.29.174.195:39869)
I20260810 18:48:56.853125 5555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed46a50fa9c543999c01dc194893f72f" candidate_uuid: "72d7366c4f5a42b5a7d1658667ab843a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db142a8d53245e2a8df5d5179eb4c4f"
I20260810 18:48:56.853183 5763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed46a50fa9c543999c01dc194893f72f" candidate_uuid: "72d7366c4f5a42b5a7d1658667ab843a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaece6422912447a90b46a3ff4c7f012"
I20260810 18:48:56.853325 5555 raft_consensus.cc:3037] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:56.853381 5763 raft_consensus.cc:3037] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:56.855646 5763 raft_consensus.cc:2454] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72d7366c4f5a42b5a7d1658667ab843a in term 1.
I20260810 18:48:56.855675 5555 raft_consensus.cc:2454] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72d7366c4f5a42b5a7d1658667ab843a in term 1.
I20260810 18:48:56.856016 5192 leader_election.cc:302] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72d7366c4f5a42b5a7d1658667ab843a, eaece6422912447a90b46a3ff4c7f012; no voters:
I20260810 18:48:56.856276 5877 raft_consensus.cc:2781] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:56.856678 5877 raft_consensus.cc:686] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 1 LEADER]: Becoming Leader. State: Replica: 72d7366c4f5a42b5a7d1658667ab843a, State: Running, Role: LEADER
I20260810 18:48:56.857097 5877 consensus_queue.cc:237] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } }
I20260810 18:48:56.861486 5063 catalog_manager.cc:5495] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a reported cstate change: term changed from 0 to 1, leader changed from <none> to 72d7366c4f5a42b5a7d1658667ab843a (127.29.174.193). New cstate: current_term: 1 leader_uuid: "72d7366c4f5a42b5a7d1658667ab843a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72d7366c4f5a42b5a7d1658667ab843a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 40445 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:56.890412 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.885209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33468 (local address 127.29.174.193:40445)
0810 18:48:56.885409 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.885415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.885523 (+ 108us) server_negotiation.cc:407] Connection header received
0810 18:48:56.885732 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.885736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.885812 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.885939 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:56.886325 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.887059 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.888371 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.888772 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.888879 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.889346 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:56.889827 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:56.889961 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.890072 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.890151 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 18:48:56.897117 5847 tablet.cc:2369] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:56.897800 5555 raft_consensus.cc:1261] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 1 FOLLOWER]: Refusing update from remote peer 72d7366c4f5a42b5a7d1658667ab843a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:56.897876 5763 raft_consensus.cc:1261] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 1 FOLLOWER]: Refusing update from remote peer 72d7366c4f5a42b5a7d1658667ab843a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:56.898692 5884 consensus_queue.cc:1035] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db142a8d53245e2a8df5d5179eb4c4f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 44859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:56.899047 5877 consensus_queue.cc:1035] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaece6422912447a90b46a3ff4c7f012" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:56.910143 5890 mvcc.cc:205] Tried to move back new op lower bound from 7317044170324627456 to 7317044170169921536. Current Snapshot: MvccSnapshot[applied={T|T < 7317044170324627456}]
I20260810 18:48:56.923334 5063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:56.923636 5063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:56.927486 5063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0e10f9064d5472bbec2850532f3777a: Sending DeleteTablet for 3 replicas of tablet ed46a50fa9c543999c01dc194893f72f
I20260810 18:48:56.928623 5526 tablet_service.cc:1514] Processing DeleteTablet for tablet ed46a50fa9c543999c01dc194893f72f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:56 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:52926
I20260810 18:48:56.928627 5729 tablet_service.cc:1514] Processing DeleteTablet for tablet ed46a50fa9c543999c01dc194893f72f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:56 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:57450
I20260810 18:48:56.928679 5286 tablet_service.cc:1514] Processing DeleteTablet for tablet ed46a50fa9c543999c01dc194893f72f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:56 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:33458
I20260810 18:48:56.929557 5902 tablet_replica.cc:330] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: stopping tablet replica
I20260810 18:48:56.929965 5902 raft_consensus.cc:2229] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:56.930493 5902 raft_consensus.cc:2258] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:56.930907 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 5165
I20260810 18:48:56.932668 5902 ts_tablet_manager.cc:1905] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:56.938282 5905 tablet_replica.cc:330] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: stopping tablet replica
I20260810 18:48:56.938832 5905 raft_consensus.cc:2229] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:56.939505 5905 raft_consensus.cc:2258] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:56.939944 5902 ts_tablet_manager.cc:1918] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:56.940099 5902 log.cc:1196] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/ed46a50fa9c543999c01dc194893f72f
I20260810 18:48:56.940657 5902 ts_tablet_manager.cc:1939] T ed46a50fa9c543999c01dc194893f72f P 5db142a8d53245e2a8df5d5179eb4c4f: Deleting consensus metadata
I20260810 18:48:56.941773 5905 ts_tablet_manager.cc:1905] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:56.942366 5904 tablet_replica.cc:330] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: stopping tablet replica
I20260810 18:48:56.942509 5037 catalog_manager.cc:4841] TS 5db142a8d53245e2a8df5d5179eb4c4f (127.29.174.194:44859): tablet ed46a50fa9c543999c01dc194893f72f (table test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]) successfully deleted
I20260810 18:48:56.942883 5904 raft_consensus.cc:2229] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:56.943404 5904 raft_consensus.cc:2258] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:56.945186 5904 ts_tablet_manager.cc:1905] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:56.949998 5905 ts_tablet_manager.cc:1918] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:56.950188 5905 log.cc:1196] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/ed46a50fa9c543999c01dc194893f72f
I20260810 18:48:56.950846 5905 ts_tablet_manager.cc:1939] T ed46a50fa9c543999c01dc194893f72f P 72d7366c4f5a42b5a7d1658667ab843a: Deleting consensus metadata
I20260810 18:48:56.952404 5904 ts_tablet_manager.cc:1918] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:56.952569 5904 log.cc:1196] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/ed46a50fa9c543999c01dc194893f72f
I20260810 18:48:56.952631 5042 catalog_manager.cc:4841] TS 72d7366c4f5a42b5a7d1658667ab843a (127.29.174.193:40445): tablet ed46a50fa9c543999c01dc194893f72f (table test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]) successfully deleted
I20260810 18:48:56.953155 5904 ts_tablet_manager.cc:1939] T ed46a50fa9c543999c01dc194893f72f P eaece6422912447a90b46a3ff4c7f012: Deleting consensus metadata
Aug 10 18:48:56 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:48:56.955049 5040 catalog_manager.cc:4841] TS eaece6422912447a90b46a3ff4c7f012 (127.29.174.195:39869): tablet ed46a50fa9c543999c01dc194893f72f (table test-table [id=294f38f7e0bb49098f8b86cf15f8a13d]) successfully deleted
I20260810 18:48:56.959949 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:56.932250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.29.174.193:40445)
0810 18:48:56.933980 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:56.933986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:56.943184 (+ 9198us) server_negotiation.cc:407] Connection header received
0810 18:48:56.943587 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:56.943593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:56.943701 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:56.943776 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:56.945417 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.946616 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.948321 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:56.949015 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:56.949103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:56.954333 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:56.954373 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:56.954390 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:56.954429 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:56.957315 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.957933 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.957942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.957949 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.958092 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:56.958489 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:56.958494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:56.958498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:56.958834 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:56.959131 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:56.959410 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 18:48:56.959635 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1612}
I20260810 18:48:56.960559 5395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:57.054616 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 5165
I20260810 18:48:57.079828 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 5417
Aug 10 18:48:57 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:48:57.093983 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.080594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.29.174.194:44859)
0810 18:48:57.080801 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.080805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.080824 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:57.080945 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.080950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.081025 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.081128 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.082292 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.083100 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.084314 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.084685 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.084774 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.088672 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:57.088683 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:57.088697 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:57.088731 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:57.091355 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:57.091972 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:57.091979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:57.091982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:57.092107 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:57.092463 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:57.092467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:57.092470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:57.092859 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:57.093129 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:57.093430 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:48:57.093643 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:48:57.094732 5621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:57.180186 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 5417
I20260810 18:48:57.200582 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 5636
Aug 10 18:48:57 dist-test-slave-6zbq krb5kdc[4993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:48:57.214305 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.201333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57466 (local address 127.29.174.195:39869)
0810 18:48:57.201515 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.201519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.201538 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:57.201655 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.201658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.201737 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.201825 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.203075 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.203801 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.204978 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:57.205346 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:57.205431 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:57.209416 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:57.209429 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:57.209445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:57.209478 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:57.211974 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:57.212558 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:57.212564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:57.212566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:57.212707 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:57.213027 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:57.213030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:57.213032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:57.213364 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:57.213566 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:57.213841 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 18:48:57.214007 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:48:57.214890 5836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:57.303707 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 5636
I20260810 18:48:57.323325 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 5010
I20260810 18:48:57.324445 5116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:57.429531 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 5010
2026-08-10T18:48:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4143 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T18:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:57Z Disabled control of system clock
I20260810 18:48:57.502743 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35851
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:48:57.810290 5922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:57.810557 5922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:57.810686 5922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:57.817564 5922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:57.817667 5922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:57.817710 5922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:57.817788 5922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:57.817840 5922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:57.826865 5922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:57.828048 5922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:57.829794 5922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:57.840875 5928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:57.840967 5929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:57.840978 5932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:57.842316 5922 server_base.cc:1034] running on GCE node
I20260810 18:48:57.843104 5922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:57.844427 5922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:57.845623 5922 hybrid_clock.cc:648] HybridClock initialized: now 1786387737845587 us; error 30 us; skew 500 ppm
I20260810 18:48:57.845953 5922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:48:57.848109 5922 webserver.cc:460] Webserver started at http://127.29.174.254:38267/ using document root <none> and password file <none>
I20260810 18:48:57.848747 5922 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:57.848840 5922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:57.849144 5922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:57.851854 5922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9"
format_stamp: "Formatted at 2026-08-10 18:48:57 on dist-test-slave-6zbq"
server_key: "799aef2ba47c0fd2963d1d1424e397af"
server_key_iv: "673e46b597a4d3888c92df0d9fa7b3d7"
server_key_version: "encryptionkey@0"
I20260810 18:48:57.852594 5922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9"
format_stamp: "Formatted at 2026-08-10 18:48:57 on dist-test-slave-6zbq"
server_key: "799aef2ba47c0fd2963d1d1424e397af"
server_key_iv: "673e46b597a4d3888c92df0d9fa7b3d7"
server_key_version: "encryptionkey@0"
I20260810 18:48:57.857265 5922 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:48:57.860373 5941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:57.861543 5922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:57.861688 5922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9"
format_stamp: "Formatted at 2026-08-10 18:48:57 on dist-test-slave-6zbq"
server_key: "799aef2ba47c0fd2963d1d1424e397af"
server_key_iv: "673e46b597a4d3888c92df0d9fa7b3d7"
server_key_version: "encryptionkey@0"
I20260810 18:48:57.861872 5922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:57.867185 5922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:57.867853 5922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:57.868109 5922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:57.884233 5922 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35851
I20260810 18:48:57.884268 6024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35851 every 8 connection(s)
I20260810 18:48:57.885386 5922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:48:57.888674 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 5922
I20260810 18:48:57.888713 6028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:57.888837 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:48:57.889442 30395 external_mini_cluster.cc:1428] Setting key 53b0c5018e5625f8bc17373e0ec9bd85
I20260810 18:48:57.897198 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:57.891236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.29.174.254:35851)
0810 18:48:57.892173 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:57.892182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:57.892226 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:48:57.894040 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:57.894060 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:57.894472 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:57.894792 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:57.895788 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:57.895795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:57.895806 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:57.895818 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:57.895881 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:57.895895 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:57.896046 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:57.896160 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:48:57.896371 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":358,"threads_started":1}
I20260810 18:48:57.900225 6028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: Bootstrap starting.
I20260810 18:48:57.902907 6028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:57.903808 6028 log.cc:824] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:57.906462 6028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: No bootstrap required, opened a new log
I20260810 18:48:57.913117 6028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } }
I20260810 18:48:57.913422 6028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:57.913517 6028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ede0ee35c13420bbc41fe5dc3a2f9e9, State: Initialized, Role: FOLLOWER
I20260810 18:48:57.914062 6028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } }
I20260810 18:48:57.914280 6028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:48:57.914427 6028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:48:57.914573 6028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:57.916651 6028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } }
I20260810 18:48:57.917129 6028 leader_election.cc:302] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ede0ee35c13420bbc41fe5dc3a2f9e9; no voters:
I20260810 18:48:57.917676 6028 leader_election.cc:288] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:48:57.917807 6033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:57.918476 6033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [term 1 LEADER]: Becoming Leader. State: Replica: 2ede0ee35c13420bbc41fe5dc3a2f9e9, State: Running, Role: LEADER
I20260810 18:48:57.919006 6033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } }
I20260810 18:48:57.919574 6028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:57.921409 6035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ede0ee35c13420bbc41fe5dc3a2f9e9. Latest consensus state: current_term: 1 leader_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } } }
I20260810 18:48:57.921880 6035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:57.921448 6034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ede0ee35c13420bbc41fe5dc3a2f9e9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35851 } } }
I20260810 18:48:57.922411 6034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:57.923070 6047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:57.928167 6047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:48:57.936882 6047 catalog_manager.cc:1347] Generated new cluster ID: af6ce3e6175d4cf0a97d7ff267245898
I20260810 18:48:57.936987 6047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:57.959112 6047 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:57.959614 6047 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:57.975368 6047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: Generated new TSK 0
I20260810 18:48:57.976282 6047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:48:57.981683 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--builtin_ntp_servers=127.29.174.212:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:48:58.299067 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:58.299417 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:58.299562 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:58.306775 6066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:58.306862 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:58.306979 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:48:58.315722 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:58.316830 6066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:58.318470 6066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:58.330507 6075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:58.330576 6073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:58.330621 6077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:58.331585 6066 server_base.cc:1034] running on GCE node
I20260810 18:48:58.332115 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:58.333014 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:58.334223 6066 hybrid_clock.cc:648] HybridClock initialized: now 1786387738334179 us; error 35 us; skew 500 ppm
I20260810 18:48:58.334522 6066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:48:58.336508 6066 webserver.cc:460] Webserver started at http://127.29.174.193:33927/ using document root <none> and password file <none>
I20260810 18:48:58.337101 6066 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:58.337182 6066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:58.337450 6066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:58.340059 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "10d3d3fd4f754a56a4fa7240eb5172be"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "bc81f88da5511297e552c751c91f5d5d"
server_key_iv: "71b5472f32e6307088a9da871e94d861"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.340854 6066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "10d3d3fd4f754a56a4fa7240eb5172be"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "bc81f88da5511297e552c751c91f5d5d"
server_key_iv: "71b5472f32e6307088a9da871e94d861"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.345397 6066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:48:58.348457 6085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.349589 6066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:48:58.349722 6066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "10d3d3fd4f754a56a4fa7240eb5172be"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "bc81f88da5511297e552c751c91f5d5d"
server_key_iv: "71b5472f32e6307088a9da871e94d861"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.349887 6066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:58.354933 6066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:58.355613 6066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:58.355832 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:58.356601 6066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:58.357877 6066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:58.357960 6066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.358036 6066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:58.358088 6066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.386916 6066 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:45027
I20260810 18:48:58.386991 6275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:45027 every 8 connection(s)
I20260810 18:48:58.388052 6066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:48:58.393997 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 6066
I20260810 18:48:58.394170 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:48:58.394856 30395 external_mini_cluster.cc:1428] Setting key 96abd2a78f7b38bdcf78ed7be3357777
I20260810 18:48:58.395438 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.390278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48459 (local address 127.29.174.254:35851)
0810 18:48:58.390459 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:58.390464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:58.391622 (+ 1158us) server_negotiation.cc:407] Connection header received
0810 18:48:58.393197 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:58.393203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:58.393267 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:58.393355 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:58.394739 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:58.394744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:58.394747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:58.394764 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:58.394793 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:58.394804 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:58.394909 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:58.395096 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:48:58.395161 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:48:58.396229 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.390662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35851 (local address 127.29.174.193:48459)
0810 18:48:58.391488 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:58.391520 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:58.392950 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:58.393604 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:58.393615 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:58.393730 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:58.394452 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:58.394455 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:58.394489 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:58.394568 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:58.394893 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:58.394904 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:58.394911 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:48:58.395150 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 18:48:58.395372 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":253,"threads_started":1}
I20260810 18:48:58.397629 6276 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35851
I20260810 18:48:58.397879 6276 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:58.397855 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--builtin_ntp_servers=127.29.174.212:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:48:58.398428 6276 heartbeater.cc:511] Master 127.29.174.254:35851 requested a full tablet report, sending...
I20260810 18:48:58.400421 5969 ts_manager.cc:194] Registered new tserver with Master: 10d3d3fd4f754a56a4fa7240eb5172be (127.29.174.193:45027)
W20260810 18:48:58.715086 6279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:58.715350 6279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:58.715476 6279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:58.722599 6279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:58.722692 6279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:58.722829 6279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:48:58.731598 6279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:58.732752 6279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:58.734436 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:58.746675 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:58.746596 6288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:58.746645 6286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:58.747641 6279 server_base.cc:1034] running on GCE node
I20260810 18:48:58.748204 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:58.749074 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:58.750269 6279 hybrid_clock.cc:648] HybridClock initialized: now 1786387738750219 us; error 43 us; skew 500 ppm
I20260810 18:48:58.750581 6279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:48:58.752596 6279 webserver.cc:460] Webserver started at http://127.29.174.194:38745/ using document root <none> and password file <none>
I20260810 18:48:58.753260 6279 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:58.753362 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:58.753672 6279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:58.756453 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "e0ccf26001c44498a5379dc728ef25c8"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "75561ac1f2ab0fb9f96c321fbd8f0cd5"
server_key_iv: "f0970c09943af015d8a9f025e64cfca8"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.757280 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "e0ccf26001c44498a5379dc728ef25c8"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "75561ac1f2ab0fb9f96c321fbd8f0cd5"
server_key_iv: "f0970c09943af015d8a9f025e64cfca8"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.762778 6279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 18:48:58.765938 6298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.767246 6279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:48:58.767400 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "e0ccf26001c44498a5379dc728ef25c8"
format_stamp: "Formatted at 2026-08-10 18:48:58 on dist-test-slave-6zbq"
server_key: "75561ac1f2ab0fb9f96c321fbd8f0cd5"
server_key_iv: "f0970c09943af015d8a9f025e64cfca8"
server_key_version: "encryptionkey@0"
I20260810 18:48:58.767591 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:58.772748 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:58.773442 6279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:58.773687 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:58.774535 6279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:58.775794 6279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:58.775887 6279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.775991 6279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:58.776062 6279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:58.805649 6279 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:36111
I20260810 18:48:58.805713 6487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:36111 every 8 connection(s)
I20260810 18:48:58.806902 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:48:58.808454 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 6279
I20260810 18:48:58.808624 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:48:58.809119 30395 external_mini_cluster.cc:1428] Setting key 5f7c30ebd8812593d346183597a526ff
I20260810 18:48:58.812011 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--builtin_ntp_servers=127.29.174.212:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:48:58.814852 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.809430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:43539 (local address 127.29.174.254:35851)
0810 18:48:58.809651 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:58.809656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:58.811186 (+ 1530us) server_negotiation.cc:407] Connection header received
0810 18:48:58.812819 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:58.812824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:58.812888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:58.812995 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:58.814140 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:58.814147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:58.814151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:58.814170 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:58.814207 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:58.814220 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:58.814321 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:58.814515 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:48:58.814577 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:58.815635 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:58.809990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35851 (local address 127.29.174.194:43539)
0810 18:48:58.811019 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:58.811073 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:58.812556 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:58.813155 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:58.813162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:58.813244 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:58.813850 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:58.813853 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:58.813888 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:58.813970 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:58.814307 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:58.814312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:58.814317 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:48:58.814553 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 18:48:58.814750 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":314,"threads_started":1}
I20260810 18:48:58.817070 6488 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35851
I20260810 18:48:58.817353 6488 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:58.817893 6488 heartbeater.cc:511] Master 127.29.174.254:35851 requested a full tablet report, sending...
I20260810 18:48:58.819244 5969 ts_manager.cc:194] Registered new tserver with Master: e0ccf26001c44498a5379dc728ef25c8 (127.29.174.194:36111)
W20260810 18:48:59.135908 6495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:59.136195 6495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:59.136332 6495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:59.143533 6495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:59.143625 6495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:59.143762 6495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:48:59.152686 6495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:59.153934 6495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:59.155663 6495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:59.168089 6503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:59.168159 6507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:59.168121 6502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:59.169340 6495 server_base.cc:1034] running on GCE node
I20260810 18:48:59.169860 6495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:59.170833 6495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:59.172042 6495 hybrid_clock.cc:648] HybridClock initialized: now 1786387739171992 us; error 43 us; skew 500 ppm
I20260810 18:48:59.172340 6495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:48:59.174369 6495 webserver.cc:460] Webserver started at http://127.29.174.195:43479/ using document root <none> and password file <none>
I20260810 18:48:59.174989 6495 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:59.175084 6495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:59.175414 6495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:59.178246 6495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "c0b32066be5345e798a161901658d060"
format_stamp: "Formatted at 2026-08-10 18:48:59 on dist-test-slave-6zbq"
server_key: "e4ebe49fd3d9ca52000ee1f6262ce276"
server_key_iv: "e5945dfb89903a505b0ac9d0bc5f71d8"
server_key_version: "encryptionkey@0"
I20260810 18:48:59.179080 6495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "c0b32066be5345e798a161901658d060"
format_stamp: "Formatted at 2026-08-10 18:48:59 on dist-test-slave-6zbq"
server_key: "e4ebe49fd3d9ca52000ee1f6262ce276"
server_key_iv: "e5945dfb89903a505b0ac9d0bc5f71d8"
server_key_version: "encryptionkey@0"
I20260810 18:48:59.184041 6495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:48:59.187213 6520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:59.188385 6495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:48:59.188532 6495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "c0b32066be5345e798a161901658d060"
format_stamp: "Formatted at 2026-08-10 18:48:59 on dist-test-slave-6zbq"
server_key: "e4ebe49fd3d9ca52000ee1f6262ce276"
server_key_iv: "e5945dfb89903a505b0ac9d0bc5f71d8"
server_key_version: "encryptionkey@0"
I20260810 18:48:59.188720 6495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:59.193929 6495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:59.194691 6495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:59.194927 6495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:59.195747 6495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:48:59.197000 6495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:48:59.197093 6495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:59.197212 6495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:48:59.197281 6495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:59.227357 6495 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:41389
I20260810 18:48:59.227407 6712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:41389 every 8 connection(s)
I20260810 18:48:59.228586 6495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:48:59.234272 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 6495
I20260810 18:48:59.234462 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:48:59.234902 30395 external_mini_cluster.cc:1428] Setting key cec1ceb5f9f3e0782a24cbdc0c06c85c
I20260810 18:48:59.236635 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.230954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:41159 (local address 127.29.174.254:35851)
0810 18:48:59.231141 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.231146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.232519 (+ 1373us) server_negotiation.cc:407] Connection header received
0810 18:48:59.234433 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.234439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.234505 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.234611 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.235895 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.235899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.235903 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.235918 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.235949 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.235961 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.236089 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.236299 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.236364 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:48:59.237468 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.231413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35851 (local address 127.29.174.195:41159)
0810 18:48:59.232354 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.232391 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.234014 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.234833 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.234841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.234924 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.235588 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.235591 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.235638 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.235735 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.236072 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.236082 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.236090 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.236354 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.236582 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260810 18:48:59.238705 6713 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35851
I20260810 18:48:59.238957 6713 heartbeater.cc:464] Registering TS with master...
I20260810 18:48:59.239486 6713 heartbeater.cc:511] Master 127.29.174.254:35851 requested a full tablet report, sending...
I20260810 18:48:59.240813 5969 ts_manager.cc:194] Registered new tserver with Master: c0b32066be5345e798a161901658d060 (127.29.174.195:41389)
I20260810 18:48:59.249960 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:48:59.256860 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.254292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.29.174.254:35851)
0810 18:48:59.254467 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.254471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.255205 (+ 734us) server_negotiation.cc:407] Connection header received
0810 18:48:59.255352 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.255356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.255406 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.255485 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.256319 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.256324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.256327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.256343 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.256374 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.256385 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.256488 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.256540 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.256582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:48:59.260604 5969 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:48:59.267380 5969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:48:59.287256 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.279218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60770)
0810 18:48:59.283567 (+ 4349us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.283583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.283776 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.284714 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.284719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.284746 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.285914 (+ 1168us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.285917 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.285932 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.285972 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.286405 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.286418 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.286426 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.286602 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.286693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4203,"thread_start_us":673,"threads_started":1}
I20260810 18:48:59.287285 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.278674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:41389 (local address 127.0.0.1:39228)
0810 18:48:59.279871 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.279919 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.280042 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.284583 (+ 4541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.284592 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.284699 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.285600 (+ 901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.285603 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.285643 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.285726 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.286637 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.286642 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.286647 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.286767 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.286808 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":4608,"threads_started":1}
I20260810 18:48:59.287959 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.279758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.29.174.193:45027)
0810 18:48:59.283050 (+ 3292us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.283058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.283677 (+ 619us) server_negotiation.cc:407] Connection header received
0810 18:48:59.283947 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.283956 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.284203 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.284322 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.286146 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.286153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.286164 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.286183 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.286239 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.286265 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.287494 (+ 1229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.287568 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.287672 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3139,"thread_start_us":2737,"threads_started":1}
I20260810 18:48:59.288121 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.278870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.29.174.195:41389)
0810 18:48:59.284104 (+ 5234us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.284111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.284143 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:48:59.284202 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.284208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.284387 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.285203 (+ 816us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.286142 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.286150 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.286164 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.286182 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.286476 (+ 294us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.286505 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.287647 (+ 1142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.287710 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.287799 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5077,"thread_start_us":4905,"threads_started":1}
I20260810 18:48:59.288866 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.284057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50608 (local address 127.29.174.194:36111)
0810 18:48:59.284552 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.284560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.285563 (+ 1003us) server_negotiation.cc:407] Connection header received
0810 18:48:59.285799 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.285805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.285946 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.286066 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.287981 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.287988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.287997 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.288023 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.288080 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.288100 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.288253 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.288455 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.288541 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":229,"threads_started":1}
I20260810 18:48:59.288908 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.284565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:36111 (local address 127.0.0.1:50608)
0810 18:48:59.285451 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.285467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.285659 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.286086 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.286091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.286112 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.287357 (+ 1245us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.287360 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.287812 (+ 452us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.287850 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.288309 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.288314 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.288319 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.288421 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.288470 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":983,"threads_started":1}
I20260810 18:48:59.291839 6166 tablet_service.cc:1467] Processing CreateTablet for tablet 97e3c2e13ed04c409541ba3005562f9b (DEFAULT_TABLE table=test-table [id=9fcdbd0d70174c929221fb53901fb9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:59.292369 6384 tablet_service.cc:1467] Processing CreateTablet for tablet 97e3c2e13ed04c409541ba3005562f9b (DEFAULT_TABLE table=test-table [id=9fcdbd0d70174c929221fb53901fb9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:59.292805 6601 tablet_service.cc:1467] Processing CreateTablet for tablet 97e3c2e13ed04c409541ba3005562f9b (DEFAULT_TABLE table=test-table [id=9fcdbd0d70174c929221fb53901fb9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:48:59.293341 6166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e3c2e13ed04c409541ba3005562f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:59.293730 6384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e3c2e13ed04c409541ba3005562f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:59.294045 6601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97e3c2e13ed04c409541ba3005562f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:59.304384 6740 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Bootstrap starting.
I20260810 18:48:59.307343 6741 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Bootstrap starting.
I20260810 18:48:59.307737 6740 tablet_bootstrap.cc:654] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:59.309281 6740 log.cc:824] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:59.309894 6743 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Bootstrap starting.
I20260810 18:48:59.310902 6741 tablet_bootstrap.cc:654] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:59.312016 6741 log.cc:824] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:59.312904 6743 tablet_bootstrap.cc:654] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:59.312943 6740 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: No bootstrap required, opened a new log
I20260810 18:48:59.313166 6740 ts_tablet_manager.cc:1397] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:48:59.313869 6743 log.cc:824] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:59.314997 6741 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: No bootstrap required, opened a new log
I20260810 18:48:59.315244 6741 ts_tablet_manager.cc:1397] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 18:48:59.316354 6743 tablet_bootstrap.cc:492] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: No bootstrap required, opened a new log
I20260810 18:48:59.316571 6743 ts_tablet_manager.cc:1397] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:48:59.323158 6740 raft_consensus.cc:348] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.323575 6740 raft_consensus.cc:374] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:59.323673 6740 raft_consensus.cc:729] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ccf26001c44498a5379dc728ef25c8, State: Initialized, Role: FOLLOWER
I20260810 18:48:59.323995 6743 raft_consensus.cc:348] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.324337 6743 raft_consensus.cc:374] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:59.324430 6743 raft_consensus.cc:729] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b32066be5345e798a161901658d060, State: Initialized, Role: FOLLOWER
I20260810 18:48:59.324282 6740 consensus_queue.cc:260] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.325004 6743 consensus_queue.cc:260] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.325090 6741 raft_consensus.cc:348] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.325574 6741 raft_consensus.cc:374] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:48:59.325680 6741 raft_consensus.cc:729] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10d3d3fd4f754a56a4fa7240eb5172be, State: Initialized, Role: FOLLOWER
I20260810 18:48:59.326198 6713 heartbeater.cc:503] Master 127.29.174.254:35851 was elected leader, sending a full tablet report...
I20260810 18:48:59.326366 6741 consensus_queue.cc:260] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.326833 6743 ts_tablet_manager.cc:1428] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260810 18:48:59.327754 6276 heartbeater.cc:503] Master 127.29.174.254:35851 was elected leader, sending a full tablet report...
I20260810 18:48:59.328370 6741 ts_tablet_manager.cc:1428] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 18:48:59.329560 6488 heartbeater.cc:503] Master 127.29.174.254:35851 was elected leader, sending a full tablet report...
I20260810 18:48:59.330171 6740 ts_tablet_manager.cc:1428] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260810 18:48:59.370664 6752 raft_consensus.cc:482] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:48:59.370959 6752 raft_consensus.cc:504] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.372720 6752 leader_election.cc:288] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0b32066be5345e798a161901658d060 (127.29.174.195:41389), e0ccf26001c44498a5379dc728ef25c8 (127.29.174.194:36111)
I20260810 18:48:59.375788 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.373106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:56807 (local address 127.29.174.195:41389)
0810 18:48:59.373327 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.373335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.373766 (+ 431us) server_negotiation.cc:407] Connection header received
0810 18:48:59.373918 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.373924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.373989 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.374083 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.375166 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.375173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.375176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.375192 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.375223 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.375235 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.375344 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.375460 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.375546 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:48:59.375985 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.373019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:41389 (local address 127.29.174.193:56807)
0810 18:48:59.373665 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.373680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.373802 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.374190 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.374213 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.374236 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.374949 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.374951 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.374966 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.375009 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.375334 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.375340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.375345 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.375493 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.375548 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":364,"threads_started":1}
I20260810 18:48:59.376447 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.373900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49277 (local address 127.29.174.194:36111)
0810 18:48:59.374077 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.374083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.374486 (+ 403us) server_negotiation.cc:407] Connection header received
0810 18:48:59.374590 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.374596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.374657 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.374747 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.375736 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.375742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.375746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.375764 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.375804 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.375818 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.375911 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.376156 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.376221 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:59.377012 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.373785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:36111 (local address 127.29.174.193:49277)
0810 18:48:59.374378 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.374398 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:59.374496 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:59.374804 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:59.374809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:59.374833 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:59.375483 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:48:59.375486 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:59.375503 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:48:59.375546 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:59.376015 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:59.376019 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:59.376023 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:48:59.376673 (+ 650us) client_negotiation.cc:240] Negotiation successful
0810 18:48:59.376722 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":243,"threads_started":1}
I20260810 18:48:59.377033 6634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e3c2e13ed04c409541ba3005562f9b" candidate_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b32066be5345e798a161901658d060" is_pre_election: true
I20260810 18:48:59.377525 6634 raft_consensus.cc:2454] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d3d3fd4f754a56a4fa7240eb5172be in term 0.
I20260810 18:48:59.377799 6422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e3c2e13ed04c409541ba3005562f9b" candidate_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0ccf26001c44498a5379dc728ef25c8" is_pre_election: true
I20260810 18:48:59.378121 6090 leader_election.cc:302] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d3d3fd4f754a56a4fa7240eb5172be, c0b32066be5345e798a161901658d060; no voters:
I20260810 18:48:59.378257 6422 raft_consensus.cc:2454] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10d3d3fd4f754a56a4fa7240eb5172be in term 0.
I20260810 18:48:59.378482 6752 raft_consensus.cc:2781] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:48:59.378634 6752 raft_consensus.cc:482] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:48:59.378756 6752 raft_consensus.cc:3037] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:59.380887 6752 raft_consensus.cc:504] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
I20260810 18:48:59.381470 6752 leader_election.cc:288] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [CANDIDATE]: Term 1 election: Requested vote from peers c0b32066be5345e798a161901658d060 (127.29.174.195:41389), e0ccf26001c44498a5379dc728ef25c8 (127.29.174.194:36111)
I20260810 18:48:59.381803 6634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e3c2e13ed04c409541ba3005562f9b" candidate_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b32066be5345e798a161901658d060"
I20260810 18:48:59.381914 6422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97e3c2e13ed04c409541ba3005562f9b" candidate_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0ccf26001c44498a5379dc728ef25c8"
I20260810 18:48:59.381969 6634 raft_consensus.cc:3037] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:59.382087 6422 raft_consensus.cc:3037] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:59.384094 6634 raft_consensus.cc:2454] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d3d3fd4f754a56a4fa7240eb5172be in term 1.
I20260810 18:48:59.384385 6422 raft_consensus.cc:2454] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10d3d3fd4f754a56a4fa7240eb5172be in term 1.
I20260810 18:48:59.384402 6090 leader_election.cc:302] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10d3d3fd4f754a56a4fa7240eb5172be, c0b32066be5345e798a161901658d060; no voters:
I20260810 18:48:59.384647 6752 raft_consensus.cc:2781] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:59.385030 6752 raft_consensus.cc:686] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 1 LEADER]: Becoming Leader. State: Replica: 10d3d3fd4f754a56a4fa7240eb5172be, State: Running, Role: LEADER
I20260810 18:48:59.385459 6752 consensus_queue.cc:237] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } }
W20260810 18:48:59.389967 6277 tablet.cc:2369] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:59.390563 5968 catalog_manager.cc:5495] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be reported cstate change: term changed from 0 to 1, leader changed from <none> to 10d3d3fd4f754a56a4fa7240eb5172be (127.29.174.193). New cstate: current_term: 1 leader_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10d3d3fd4f754a56a4fa7240eb5172be" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 45027 } health_report { overall_health: HEALTHY } } }
I20260810 18:48:59.408950 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.406931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.29.174.193:45027)
0810 18:48:59.407121 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.407127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.407150 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:48:59.407394 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.407398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.407471 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.407554 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.408372 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.408378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.408383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.408397 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.408426 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.408437 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.408531 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.408659 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.408714 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:48:59.416522 6422 raft_consensus.cc:1261] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 1 FOLLOWER]: Refusing update from remote peer 10d3d3fd4f754a56a4fa7240eb5172be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:59.416550 6634 raft_consensus.cc:1261] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 1 FOLLOWER]: Refusing update from remote peer 10d3d3fd4f754a56a4fa7240eb5172be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:59.417387 6755 consensus_queue.cc:1035] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0b32066be5345e798a161901658d060" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:59.417706 6752 consensus_queue.cc:1035] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0ccf26001c44498a5379dc728ef25c8" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:59.430188 6762 mvcc.cc:205] Tried to move back new op lower bound from 7317044180641632256 to 7317044180526817280. Current Snapshot: MvccSnapshot[applied={T|T < 7317044180641632256}]
I20260810 18:48:59.446071 5968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:59.446401 5968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:48:59.449952 5968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ede0ee35c13420bbc41fe5dc3a2f9e9: Sending DeleteTablet for 3 replicas of tablet 97e3c2e13ed04c409541ba3005562f9b
I20260810 18:48:59.451107 6166 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e3c2e13ed04c409541ba3005562f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:59 UTC) from {username='slave'} at 127.0.0.1:60770
I20260810 18:48:59.451340 6384 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e3c2e13ed04c409541ba3005562f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:59 UTC) from {username='slave'} at 127.0.0.1:50608
I20260810 18:48:59.451344 6601 tablet_service.cc:1514] Processing DeleteTablet for tablet 97e3c2e13ed04c409541ba3005562f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:48:59 UTC) from {username='slave'} at 127.0.0.1:39228
I20260810 18:48:59.452661 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 6066
I20260810 18:48:59.454277 6777 tablet_replica.cc:330] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: stopping tablet replica
I20260810 18:48:59.454756 6777 raft_consensus.cc:2229] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:48:59.455536 6777 raft_consensus.cc:2258] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:59.461550 6777 ts_tablet_manager.cc:1905] T 97e3c2e13ed04c409541ba3005562f9b P 10d3d3fd4f754a56a4fa7240eb5172be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:59.462746 6780 tablet_replica.cc:330] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: stopping tablet replica
I20260810 18:48:59.463229 6780 raft_consensus.cc:2229] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:59.463790 6780 raft_consensus.cc:2258] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:59.465648 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.455046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.29.174.193:45027)
0810 18:48:59.455208 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.455214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.463013 (+ 7799us) server_negotiation.cc:407] Connection header received
0810 18:48:59.463126 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.463132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.463192 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.463317 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.464808 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.464814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.464819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.464838 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.464876 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.464891 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.464982 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.465223 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.465288 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:48:59.467221 6269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:59.467801 6778 tablet_replica.cc:330] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: stopping tablet replica
I20260810 18:48:59.468211 6780 ts_tablet_manager.cc:1905] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:59.468241 6778 raft_consensus.cc:2229] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:48:59.468809 6778 raft_consensus.cc:2258] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:48:59.470911 6778 ts_tablet_manager.cc:1905] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:48:59.474597 6780 ts_tablet_manager.cc:1918] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:59.474751 6780 log.cc:1196] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/97e3c2e13ed04c409541ba3005562f9b
I20260810 18:48:59.475208 6780 ts_tablet_manager.cc:1939] T 97e3c2e13ed04c409541ba3005562f9b P e0ccf26001c44498a5379dc728ef25c8: Deleting consensus metadata
I20260810 18:48:59.476445 6778 ts_tablet_manager.cc:1918] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:48:59.476579 6778 log.cc:1196] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/97e3c2e13ed04c409541ba3005562f9b
I20260810 18:48:59.476840 5944 catalog_manager.cc:4841] TS e0ccf26001c44498a5379dc728ef25c8 (127.29.174.194:36111): tablet 97e3c2e13ed04c409541ba3005562f9b (table test-table [id=9fcdbd0d70174c929221fb53901fb9bc]) successfully deleted
I20260810 18:48:59.476982 6778 ts_tablet_manager.cc:1939] T 97e3c2e13ed04c409541ba3005562f9b P c0b32066be5345e798a161901658d060: Deleting consensus metadata
I20260810 18:48:59.478183 5944 catalog_manager.cc:4841] TS c0b32066be5345e798a161901658d060 (127.29.174.195:41389): tablet 97e3c2e13ed04c409541ba3005562f9b (table test-table [id=9fcdbd0d70174c929221fb53901fb9bc]) successfully deleted
I20260810 18:48:59.562098 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 6066
I20260810 18:48:59.582473 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 6279
W20260810 18:48:59.582473 5946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:45027 (error 108)
I20260810 18:48:59.584169 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.583751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60804)
0810 18:48:59.583908 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.584014 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:48:59.584406 5946 catalog_manager.cc:4584] TS 10d3d3fd4f754a56a4fa7240eb5172be (127.29.174.193:45027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97e3c2e13ed04c409541ba3005562f9b: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
I20260810 18:48:59.585376 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.583225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.29.174.194:36111)
0810 18:48:59.583386 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.583392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.583432 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:48:59.583624 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.583629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.583717 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.583802 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.584757 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.584762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.584765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.584781 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.584815 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.584826 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.584928 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.584998 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.585045 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:48:59.586055 6477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:59.638073 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.637481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60816)
0810 18:48:59.637737 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.637873 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 18:48:59.639110 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.638734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60830)
0810 18:48:59.638870 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.638956 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 18:48:59.672504 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 6279
I20260810 18:48:59.692735 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 6495
I20260810 18:48:59.695571 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.693619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.29.174.195:41389)
0810 18:48:59.693825 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:59.693831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:59.693850 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:59.693904 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:59.693907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:59.693985 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:59.694070 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:59.694971 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:59.694975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:59.694978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:48:59.694993 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:59.695023 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:48:59.695034 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:59.695148 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:59.695227 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 18:48:59.695296 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:48:59.696201 6705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:59.699542 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.699116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60834)
0810 18:48:59.699297 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.699410 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:48:59.700618 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.700167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60838)
0810 18:48:59.700366 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.700465 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:48:59.781453 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.780807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60842)
0810 18:48:59.781088 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.781230 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 18:48:59.783499 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:59.782644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:45027 (local address 127.0.0.1:60850)
0810 18:48:59.783010 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:59.783164 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260810 18:48:59.786561 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 6495
I20260810 18:48:59.808156 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 5922
I20260810 18:48:59.809221 6016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:59.913101 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 5922
2026-08-10T18:48:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2484 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 18:48:59.960271 30395 mini_oidc.cc:152] Starting JWKS server
I20260810 18:48:59.964428 30395 webserver.cc:460] Webserver started at https://127.0.0.1:45253/ using document root <none> and password file <none>
I20260810 18:48:59.964762 30395 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:48:59.965369 30395 webserver.cc:460] Webserver started at http://127.0.0.1:34129/ using document root <none> and password file <none>
2026-08-10T18:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:48:59Z Disabled control of system clock
I20260810 18:48:59.996898 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35025
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35025
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45253/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/testchainca.cert with env {}
W20260810 18:49:00.314110 6800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:00.314433 6800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:00.314568 6800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:00.321666 6800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:00.321772 6800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:00.321812 6800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:00.321897 6800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:00.321962 6800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:49:00.322038 6800 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:49:00.322094 6800 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45253/jwks.json
I20260810 18:49:00.332067 6800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45253/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:00.333258 6800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:00.334497 6800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:00.346068 6808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:00.346029 6811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:00.345999 6809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:00.347289 6800 server_base.cc:1034] running on GCE node
I20260810 18:49:00.348099 6800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:00.349472 6800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:00.350664 6800 hybrid_clock.cc:648] HybridClock initialized: now 1786387740350625 us; error 32 us; skew 500 ppm
I20260810 18:49:00.351012 6800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:00.355839 6800 webserver.cc:460] Webserver started at http://127.29.174.254:42227/ using document root <none> and password file <none>
I20260810 18:49:00.356653 6800 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:00.356756 6800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:00.357056 6800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:00.360373 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "9e0f855820574ad89e9cfa5c0c2e81f7"
format_stamp: "Formatted at 2026-08-10 18:49:00 on dist-test-slave-6zbq"
server_key: "c7ac7c6088112d43bdeadfd055a21489"
server_key_iv: "07cb685f84e9e4fee3320d71444034e5"
server_key_version: "encryptionkey@0"
I20260810 18:49:00.361236 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "9e0f855820574ad89e9cfa5c0c2e81f7"
format_stamp: "Formatted at 2026-08-10 18:49:00 on dist-test-slave-6zbq"
server_key: "c7ac7c6088112d43bdeadfd055a21489"
server_key_iv: "07cb685f84e9e4fee3320d71444034e5"
server_key_version: "encryptionkey@0"
I20260810 18:49:00.367014 6800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 18:49:00.370846 6826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:00.372125 6800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:49:00.372313 6800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "9e0f855820574ad89e9cfa5c0c2e81f7"
format_stamp: "Formatted at 2026-08-10 18:49:00 on dist-test-slave-6zbq"
server_key: "c7ac7c6088112d43bdeadfd055a21489"
server_key_iv: "07cb685f84e9e4fee3320d71444034e5"
server_key_version: "encryptionkey@0"
I20260810 18:49:00.372512 6800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:00.399442 6800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:00.400238 6800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:00.400521 6800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:00.417502 6800 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35025
I20260810 18:49:00.417523 6909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35025 every 8 connection(s)
I20260810 18:49:00.418752 6800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:00.423202 6910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:00.424911 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 6800
I20260810 18:49:00.425112 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:00.425729 30395 external_mini_cluster.cc:1428] Setting key ed86564aa23b076997c0f5fa7f883ea3
I20260810 18:49:00.436290 6910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7: Bootstrap starting.
I20260810 18:49:00.440140 6910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:00.441399 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:00.428065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.29.174.254:35025)
0810 18:49:00.429194 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:00.429220 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:00.429270 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:49:00.431521 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:00.431548 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:00.432096 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:00.432514 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:00.433043 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.435051 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.436334 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.437313 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.437497 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.438701 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:00.438712 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:00.438728 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:00.439406 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:00.439517 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:00.439543 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:00.439680 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:00.439855 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:49:00.440172 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":337,"threads_started":1}
I20260810 18:49:00.441438 6910 log.cc:824] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:00.444504 6910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7: No bootstrap required, opened a new log
I20260810 18:49:00.445319 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:49:00.451622 6910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } }
I20260810 18:49:00.451960 6910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:00.452054 6910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0f855820574ad89e9cfa5c0c2e81f7, State: Initialized, Role: FOLLOWER
I20260810 18:49:00.452658 6910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } }
I20260810 18:49:00.452919 6910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:00.453063 6910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:00.453373 6910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:00.455595 6910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } }
I20260810 18:49:00.456060 6910 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0f855820574ad89e9cfa5c0c2e81f7; no voters:
I20260810 18:49:00.456566 6910 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:00.456686 6916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:00.457713 6916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [term 1 LEADER]: Becoming Leader. State: Replica: 9e0f855820574ad89e9cfa5c0c2e81f7, State: Running, Role: LEADER
I20260810 18:49:00.458189 6916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } }
I20260810 18:49:00.458567 6910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:00.460688 6921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0f855820574ad89e9cfa5c0c2e81f7. Latest consensus state: current_term: 1 leader_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } } }
I20260810 18:49:00.461107 6921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:00.461710 6918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0f855820574ad89e9cfa5c0c2e81f7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35025 } } }
I20260810 18:49:00.462060 6918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:00.462067 6935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:00.471959 6935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:00.480782 6935 catalog_manager.cc:1347] Generated new cluster ID: 3b6093bd0d454f3293cd4206a2d9e836
I20260810 18:49:00.480885 6935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:00.491456 6935 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:00.492579 6935 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:00.500718 6935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0f855820574ad89e9cfa5c0c2e81f7: Generated new TSK 0
I20260810 18:49:00.501523 6935 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:49:00.545120 6959 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 18:49:00.545328 6959 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:49:00.546550 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:00.524515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.29.174.254:35025)
0810 18:49:00.524666 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:00.524671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:00.525108 (+ 437us) server_negotiation.cc:407] Connection header received
0810 18:49:00.525437 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:00.525443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:00.525530 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:00.525701 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:49:00.526177 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.527341 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.528745 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.529443 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.529563 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.530096 (+ 533us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:49:00.545849 (+ 15753us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:49:00.546201 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:49:00.546669 6914 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 18:49:00.548161 6955 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:00.524405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35025 (local address 127.0.0.1:36604)
0810 18:49:00.525003 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:00.525023 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:00.525162 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:00.525720 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:00.525724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:00.525734 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:49:00.526046 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.526054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.527483 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.527487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.528613 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:00.528624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:00.529617 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:00.529621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:00.529759 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:00.529816 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:49:00.546110 (+ 16294us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:49:00.546426 (+ 316us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.174.254:35025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":303,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.174.254:35025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 18:49:00.549635 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 6800
I20260810 18:49:00.550851 6903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:00.645298 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 6800
2026-08-10T18:49:00Z chronyd exiting
I20260810 18:49:00.700356 30395 test_util.cc:175] -----------------------------------------------
I20260810 18:49:00.700481 30395 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387695213813-30395-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 18:49:00.704155 30395 mini_oidc.cc:152] Starting JWKS server
I20260810 18:49:00.705837 30395 webserver.cc:460] Webserver started at https://127.0.0.1:45169/ using document root <none> and password file <none>
I20260810 18:49:00.706008 30395 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:49:00.706617 30395 webserver.cc:460] Webserver started at http://127.0.0.1:46625/ using document root <none> and password file <none>
2026-08-10T18:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:00Z Disabled control of system clock
I20260810 18:49:00.736903 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38545
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:45447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:38545
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45169/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/testchainca.cert with env {}
W20260810 18:49:01.047674 6967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:01.047964 6967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:01.048087 6967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:01.055133 6967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:01.055243 6967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:01.055310 6967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:01.055394 6967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:01.055455 6967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:49:01.055540 6967 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:49:01.055593 6967 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45169/jwks.json
I20260810 18:49:01.065325 6967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:38545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45169/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:01.066586 6967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:01.067775 6967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:01.078877 6972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:01.079161 6977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:01.079188 6975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:01.080341 6967 server_base.cc:1034] running on GCE node
I20260810 18:49:01.080981 6967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:01.082201 6967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:01.083389 6967 hybrid_clock.cc:648] HybridClock initialized: now 1786387741083342 us; error 36 us; skew 500 ppm
I20260810 18:49:01.083679 6967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:01.085625 6967 webserver.cc:460] Webserver started at http://127.29.174.254:34353/ using document root <none> and password file <none>
I20260810 18:49:01.086220 6967 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:01.086320 6967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:01.086630 6967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:01.089277 6967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "53d2694a4e6941069b4524eb5393f79e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "2541eb6dfe86b80f4e548751a4624b82"
server_key_iv: "eabadc32313b579978a02d9baf26b309"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.090037 6967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "53d2694a4e6941069b4524eb5393f79e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "2541eb6dfe86b80f4e548751a4624b82"
server_key_iv: "eabadc32313b579978a02d9baf26b309"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.094769 6967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:49:01.097711 6987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:01.098896 6967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:01.099049 6967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "53d2694a4e6941069b4524eb5393f79e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "2541eb6dfe86b80f4e548751a4624b82"
server_key_iv: "eabadc32313b579978a02d9baf26b309"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.099241 6967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:01.117844 6967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:01.118563 6967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:01.118816 6967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:01.135070 6967 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:38545
I20260810 18:49:01.135098 7070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:38545 every 8 connection(s)
I20260810 18:49:01.136211 6967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:01.139510 7072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:01.143620 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 6967
I20260810 18:49:01.143819 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:01.144618 30395 external_mini_cluster.cc:1428] Setting key 0f6bc147d4ac9225647ead7b8e4861a8
I20260810 18:49:01.150516 7072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e: Bootstrap starting.
I20260810 18:49:01.153906 7072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:01.155069 7072 log.cc:824] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:01.157819 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.146436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.29.174.254:38545)
0810 18:49:01.147356 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.147366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.147407 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:49:01.149151 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.149175 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.149610 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.149919 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:01.150530 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.151831 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.153515 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.154194 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.154330 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.155470 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:01.155477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:01.155489 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:01.156218 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:01.156317 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:01.156343 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:01.156450 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:01.156670 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 18:49:01.156950 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":336,"threads_started":1}
I20260810 18:49:01.158042 7072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e: No bootstrap required, opened a new log
I20260810 18:49:01.161319 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:49:01.165236 7072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } }
I20260810 18:49:01.165529 7072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:01.165601 7072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d2694a4e6941069b4524eb5393f79e, State: Initialized, Role: FOLLOWER
I20260810 18:49:01.166042 7072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } }
I20260810 18:49:01.166232 7072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:01.166325 7072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:01.166504 7072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:01.168439 7072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } }
I20260810 18:49:01.168834 7072 leader_election.cc:302] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53d2694a4e6941069b4524eb5393f79e; no voters:
I20260810 18:49:01.169293 7072 leader_election.cc:288] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:01.169436 7080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:01.170485 7080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [term 1 LEADER]: Becoming Leader. State: Replica: 53d2694a4e6941069b4524eb5393f79e, State: Running, Role: LEADER
I20260810 18:49:01.170996 7080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } }
I20260810 18:49:01.171214 7072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:01.173569 7081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53d2694a4e6941069b4524eb5393f79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } } }
I20260810 18:49:01.173933 7081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:01.175527 7092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:01.175446 7082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53d2694a4e6941069b4524eb5393f79e. Latest consensus state: current_term: 1 leader_uuid: "53d2694a4e6941069b4524eb5393f79e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d2694a4e6941069b4524eb5393f79e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38545 } } }
I20260810 18:49:01.175846 7082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:01.180476 7092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:01.188511 7092 catalog_manager.cc:1347] Generated new cluster ID: 0741ed5fcde145ed91d84df26579f0a7
I20260810 18:49:01.188606 7092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:01.210681 7092 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:01.212291 7092 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:01.218802 7092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53d2694a4e6941069b4524eb5393f79e: Generated new TSK 0
I20260810 18:49:01.219709 7092 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:49:01.253731 7111 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 18:49:01.253825 7111 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:49:01.255008 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.235992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.29.174.254:38545)
0810 18:49:01.236127 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.236131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.236493 (+ 362us) server_negotiation.cc:407] Connection header received
0810 18:49:01.236691 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.236700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.236799 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.236949 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:49:01.237345 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.238446 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.239755 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.240453 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.240587 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.240998 (+ 411us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:49:01.254405 (+ 13407us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45169/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:49:01.254724 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45169/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 18:49:01.255046 7110 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:01.235853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:38545 (local address 127.0.0.1:42866)
0810 18:49:01.236402 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:01.236417 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:01.236527 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:01.236938 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:01.236941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:01.236950 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:49:01.237202 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.237210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.238560 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.238564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.239613 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.239624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.240599 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.240603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.240739 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.240769 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:49:01.254598 (+ 13829us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45169/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:49:01.254777 (+ 179us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.174.254:38545: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45169/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
W20260810 18:49:01.255123 7078 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45169/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260810 18:49:01.256680 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 6967
I20260810 18:49:01.257587 7060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:01.346817 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 6967
2026-08-10T18:49:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (701 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 18:49:01.405268 30395 mini_oidc.cc:152] Starting JWKS server
I20260810 18:49:01.407074 30395 webserver.cc:460] Webserver started at https://127.0.0.1:40211/ using document root <none> and password file <none>
I20260810 18:49:01.407306 30395 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:49:01.407893 30395 webserver.cc:460] Webserver started at http://127.0.0.1:33373/ using document root <none> and password file <none>
2026-08-10T18:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:01Z Disabled control of system clock
I20260810 18:49:01.436823 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38021
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:34937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:38021
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40211/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:49:01.744299 7123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:01.744620 7123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:01.744746 7123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:01.751879 7123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:01.751982 7123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:01.752049 7123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:01.752146 7123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:01.752203 7123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:49:01.752288 7123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:49:01.752341 7123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40211/jwks.json
I20260810 18:49:01.761189 7123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:38021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40211/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:01.762420 7123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:01.764148 7123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:01.775420 7129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:01.775458 7133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:01.775382 7130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:01.776612 7123 server_base.cc:1034] running on GCE node
I20260810 18:49:01.777318 7123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:01.778678 7123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:01.779889 7123 hybrid_clock.cc:648] HybridClock initialized: now 1786387741779842 us; error 35 us; skew 500 ppm
I20260810 18:49:01.780224 7123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:01.782375 7123 webserver.cc:460] Webserver started at http://127.29.174.254:40089/ using document root <none> and password file <none>
I20260810 18:49:01.783150 7123 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:01.783265 7123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:01.783593 7123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:01.786417 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "967dd4bafdfd45bc81ce717b82d6e66e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "bc241832014b0f678d70c479cbdfac8b"
server_key_iv: "b3917b7c1af2a1700fdee099bae5cc44"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.787207 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "967dd4bafdfd45bc81ce717b82d6e66e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "bc241832014b0f678d70c479cbdfac8b"
server_key_iv: "b3917b7c1af2a1700fdee099bae5cc44"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.792259 7123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:49:01.795466 7142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:01.796623 7123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:49:01.796787 7123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "967dd4bafdfd45bc81ce717b82d6e66e"
format_stamp: "Formatted at 2026-08-10 18:49:01 on dist-test-slave-6zbq"
server_key: "bc241832014b0f678d70c479cbdfac8b"
server_key_iv: "b3917b7c1af2a1700fdee099bae5cc44"
server_key_version: "encryptionkey@0"
I20260810 18:49:01.796974 7123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:01.828941 7123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:01.829742 7123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:01.830022 7123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:01.846733 7123 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:38021
I20260810 18:49:01.846768 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:38021 every 8 connection(s)
I20260810 18:49:01.847930 7123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:01.851329 7236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:01.852710 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 7123
I20260810 18:49:01.852922 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:01.853729 30395 external_mini_cluster.cc:1428] Setting key 960e32182b61254da75aee53e1f586a1
I20260810 18:49:01.864481 7236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e: Bootstrap starting.
I20260810 18:49:01.866540 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.855524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.29.174.254:38021)
0810 18:49:01.856495 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.856505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.856543 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:49:01.858077 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.858101 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.858557 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.858848 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:01.859352 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.860540 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.862385 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.863012 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.863168 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.864342 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:01.864349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:01.864361 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:01.864913 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:01.864991 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:01.865015 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:01.865103 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:01.865331 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 18:49:01.865595 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":364,"threads_started":1}
I20260810 18:49:01.867556 7236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:01.868535 7236 log.cc:824] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:01.870205 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:49:01.871488 7236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e: No bootstrap required, opened a new log
I20260810 18:49:01.878474 7236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } }
I20260810 18:49:01.878783 7236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:01.878851 7236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967dd4bafdfd45bc81ce717b82d6e66e, State: Initialized, Role: FOLLOWER
I20260810 18:49:01.879281 7236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } }
I20260810 18:49:01.879490 7236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:01.879623 7236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:01.879809 7236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:01.881820 7236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } }
I20260810 18:49:01.882261 7236 leader_election.cc:302] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 967dd4bafdfd45bc81ce717b82d6e66e; no voters:
I20260810 18:49:01.882807 7236 leader_election.cc:288] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:01.882911 7243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:01.883611 7243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [term 1 LEADER]: Becoming Leader. State: Replica: 967dd4bafdfd45bc81ce717b82d6e66e, State: Running, Role: LEADER
I20260810 18:49:01.884085 7243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } }
I20260810 18:49:01.884598 7236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:01.886791 7244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } } }
I20260810 18:49:01.887107 7244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:01.886862 7245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 967dd4bafdfd45bc81ce717b82d6e66e. Latest consensus state: current_term: 1 leader_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "967dd4bafdfd45bc81ce717b82d6e66e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38021 } } }
I20260810 18:49:01.887255 7245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:01.888055 7256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:01.893646 7256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:01.903800 7256 catalog_manager.cc:1347] Generated new cluster ID: 8f629b254aec42ebbbf89fe852e2143d
I20260810 18:49:01.903906 7256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:01.956007 7256 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:01.957324 7256 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:01.966017 7256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 967dd4bafdfd45bc81ce717b82d6e66e: Generated new TSK 0
I20260810 18:49:01.966888 7256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:01.994557 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:01.986278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.29.174.254:38021)
0810 18:49:01.986488 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:01.986495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:01.986911 (+ 416us) server_negotiation.cc:407] Connection header received
0810 18:49:01.987103 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:01.987109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:01.987196 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:01.987355 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:49:01.987760 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.988870 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.990261 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.990973 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.991106 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.991617 (+ 511us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:49:01.993887 (+ 2270us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:49:01.994202 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:49:01.994689 7239 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 18:49:01.996233 7287 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:01.986199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:38021 (local address 127.0.0.1:49172)
0810 18:49:01.986817 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:01.986833 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:01.986943 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:01.987352 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:01.987355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:01.987363 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:49:01.987618 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.987627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.989027 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.989031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.990125 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:01.990137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:01.991191 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:01.991196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:01.991329 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:01.991363 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:49:01.994098 (+ 2735us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:49:01.995804 (+ 1706us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.174.254:38021: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40211/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1428,"thread_start_us":338,"threads_started":1}
I20260810 18:49:01.998055 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 7123
I20260810 18:49:01.999191 7230 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 18:49:02.006016 7289 webserver.cc:533] Webserver: sslize failed:
E20260810 18:49:02.006105 7289 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:49:02.090297 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 7123
2026-08-10T18:49:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (741 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T18:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:02Z Disabled control of system clock
I20260810 18:49:02.177081 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45987
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45987
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/testchainca.cert with env {}
W20260810 18:49:02.491458 7297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:02.491721 7297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:02.491816 7297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:02.498757 7297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:02.498863 7297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:02.498901 7297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:02.498957 7297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:02.499004 7297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:49:02.499059 7297 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:49:02.499104 7297 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 18:49:02.508622 7297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:02.509733 7297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:02.511271 7297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:02.522168 7308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:02.522189 7309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:02.522675 7311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:02.523124 7297 server_base.cc:1034] running on GCE node
I20260810 18:49:02.523825 7297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:02.525069 7297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:02.526269 7297 hybrid_clock.cc:648] HybridClock initialized: now 1786387742526226 us; error 29 us; skew 500 ppm
I20260810 18:49:02.526768 7297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:02.528870 7297 webserver.cc:460] Webserver started at http://127.29.174.254:34073/ using document root <none> and password file <none>
I20260810 18:49:02.529488 7297 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:02.529588 7297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:02.529871 7297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:02.532612 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "9a2fdb40ef3740f49af692881a2cb9f1"
format_stamp: "Formatted at 2026-08-10 18:49:02 on dist-test-slave-6zbq"
server_key: "b5f581ca3748b2415e5ca1dd4b44582e"
server_key_iv: "ba3c8810b13efe4b72754ba065916141"
server_key_version: "encryptionkey@0"
I20260810 18:49:02.533381 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "9a2fdb40ef3740f49af692881a2cb9f1"
format_stamp: "Formatted at 2026-08-10 18:49:02 on dist-test-slave-6zbq"
server_key: "b5f581ca3748b2415e5ca1dd4b44582e"
server_key_iv: "ba3c8810b13efe4b72754ba065916141"
server_key_version: "encryptionkey@0"
I20260810 18:49:02.538097 7297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 18:49:02.541168 7322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:02.542456 7297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:02.542609 7297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "9a2fdb40ef3740f49af692881a2cb9f1"
format_stamp: "Formatted at 2026-08-10 18:49:02 on dist-test-slave-6zbq"
server_key: "b5f581ca3748b2415e5ca1dd4b44582e"
server_key_iv: "ba3c8810b13efe4b72754ba065916141"
server_key_version: "encryptionkey@0"
I20260810 18:49:02.542794 7297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:02.556280 7297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:02.556926 7297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:02.557164 7297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:02.578250 7297 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:45987
I20260810 18:49:02.578290 7401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:45987 every 8 connection(s)
I20260810 18:49:02.579604 7297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:02.583060 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 7297
I20260810 18:49:02.583258 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:02.583884 30395 external_mini_cluster.cc:1428] Setting key 9fdfabe01d62986b74768bf7616e7204
I20260810 18:49:02.584406 7402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:02.599596 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:02.586488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.29.174.254:45987)
0810 18:49:02.587647 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:02.587657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:02.587697 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:49:02.590399 (+ 2702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:02.590425 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:02.590892 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:02.591300 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:02.591858 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.593306 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.594875 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.595508 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.595670 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.596876 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:02.596885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:02.596897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:02.597404 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:02.597483 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:02.597503 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:02.598388 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:02.598496 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 18:49:02.598754 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":534,"threads_started":1}
I20260810 18:49:02.600649 7402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1: Bootstrap starting.
I20260810 18:49:02.603142 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:49:02.604378 7402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:02.605636 7402 log.cc:824] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:02.608954 7402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1: No bootstrap required, opened a new log
I20260810 18:49:02.617992 7402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } }
I20260810 18:49:02.618415 7402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:02.618506 7402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2fdb40ef3740f49af692881a2cb9f1, State: Initialized, Role: FOLLOWER
I20260810 18:49:02.619077 7402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } }
I20260810 18:49:02.619288 7402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:02.619412 7402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:02.619575 7402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:02.621627 7402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } }
I20260810 18:49:02.622113 7402 leader_election.cc:302] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a2fdb40ef3740f49af692881a2cb9f1; no voters:
I20260810 18:49:02.622723 7402 leader_election.cc:288] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:02.622901 7411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:02.623558 7411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [term 1 LEADER]: Becoming Leader. State: Replica: 9a2fdb40ef3740f49af692881a2cb9f1, State: Running, Role: LEADER
I20260810 18:49:02.624135 7411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } }
I20260810 18:49:02.625689 7402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:02.627025 7414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a2fdb40ef3740f49af692881a2cb9f1. Latest consensus state: current_term: 1 leader_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } } }
I20260810 18:49:02.627447 7414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:02.627892 7415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2fdb40ef3740f49af692881a2cb9f1" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45987 } } }
I20260810 18:49:02.628278 7415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:02.631050 7419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:02.636828 7419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:02.649423 7419 catalog_manager.cc:1347] Generated new cluster ID: 7c5534ff50eb4a7aa1ad4976d4d2dbb0
I20260810 18:49:02.649518 7419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:02.681358 7419 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:02.682670 7419 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:02.688247 7419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a2fdb40ef3740f49af692881a2cb9f1: Generated new TSK 0
I20260810 18:49:02.688942 7419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:02.792774 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:02.702978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.29.174.254:45987)
0810 18:49:02.703146 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:02.703152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:02.703617 (+ 465us) server_negotiation.cc:407] Connection header received
0810 18:49:02.703794 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:02.703801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:02.703883 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:02.704040 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:49:02.704434 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.705528 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.706948 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.707630 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.707755 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.708199 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:49:02.792016 (+ 83817us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:49:02.792458 (+ 442us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56942: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 18:49:02.792852 7446 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:02.702949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45987 (local address 127.0.0.1:56942)
0810 18:49:02.703520 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:02.703536 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:02.703648 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:02.704036 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:02.704039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:02.704048 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:49:02.704303 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.704311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.705646 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.705649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.706814 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:02.706826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:02.707799 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:02.707803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:02.707921 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:02.707953 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:49:02.792316 (+ 84363us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:49:02.792557 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.174.254:45987: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":292,"threads_started":1}
W20260810 18:49:02.792907 7407 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56942: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 18:49:02.793980 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 7297
I20260810 18:49:02.795243 7396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:02.893134 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 7297
2026-08-10T18:49:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (786 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (394 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (404 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:03 dist-test-slave-6zbq krb5kdc[7473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:03 dist-test-slave-6zbq krb5kdc[7473](info): set up 2 sockets
Aug 10 18:49:03 dist-test-slave-6zbq krb5kdc[7473](info): commencing operation
krb5kdc: starting...
W20260810 18:49:04.900502 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:05 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:49:05.101892 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39525
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:05.410482 7492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:05.410740 7492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:05.410873 7492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:05.417822 7492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:05.417922 7492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:05.417994 7492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:05.418072 7492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:05.418130 7492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:05.428346 7492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:05.429627 7492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:05.430895 7492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:05.441866 7505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:05.441933 7503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:05.441932 7507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:05.442876 7492 server_base.cc:1034] running on GCE node
I20260810 18:49:05.443576 7492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:05.444810 7492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:05.446005 7492 hybrid_clock.cc:648] HybridClock initialized: now 1786387745445956 us; error 39 us; skew 500 ppm
I20260810 18:49:05.446311 7492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:05 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387745, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:05.452554 7492 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:49:05.453999 7492 webserver.cc:460] Webserver started at http://127.29.174.254:42569/ using document root <none> and password file <none>
I20260810 18:49:05.454684 7492 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:05.454805 7492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:05.455094 7492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:05.457918 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "e7da8ab9215743c190c018b4d18715c7"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "ada0846b6100ef549444a4b7fde61028"
server_key_iv: "db3dbff483c75e2b9bb11a15184838fa"
server_key_version: "encryptionkey@0"
I20260810 18:49:05.458695 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "e7da8ab9215743c190c018b4d18715c7"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "ada0846b6100ef549444a4b7fde61028"
server_key_iv: "db3dbff483c75e2b9bb11a15184838fa"
server_key_version: "encryptionkey@0"
I20260810 18:49:05.463507 7492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:05.466703 7515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:05.467909 7492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 18:49:05.468065 7492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "e7da8ab9215743c190c018b4d18715c7"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "ada0846b6100ef549444a4b7fde61028"
server_key_iv: "db3dbff483c75e2b9bb11a15184838fa"
server_key_version: "encryptionkey@0"
I20260810 18:49:05.468259 7492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:05.480656 7492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:05.484683 7492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:05.484933 7492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:05.502264 7492 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39525
I20260810 18:49:05.502295 7602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39525 every 8 connection(s)
I20260810 18:49:05.503559 7492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:05.507061 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:05.508222 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 7492
I20260810 18:49:05.508411 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:05.509043 30395 external_mini_cluster.cc:1428] Setting key 878aae414b2ac57ebe6e8e9dd7cc3a02
I20260810 18:49:05.520567 7603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7: Bootstrap starting.
Aug 10 18:49:05 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:05.523847 7603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:05.524843 7603 log.cc:824] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:05.528064 7603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7: No bootstrap required, opened a new log
I20260810 18:49:05.531599 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:05.510938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.29.174.254:39525)
0810 18:49:05.511733 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:05.511748 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:05.511795 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:49:05.513442 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:05.513467 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:05.513919 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:05.514330 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:05.515503 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.517416 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.518616 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:05.519590 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:05.519755 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:05.523775 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:05.523786 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:05.523800 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:05.523837 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:05.527251 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:05.527939 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:05.527947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:05.527955 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:05.528197 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:05.528558 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:05.528563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:05.528566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:05.529251 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:05.529533 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:05.529852 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 18:49:05.530296 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":326,"threads_started":1}
I20260810 18:49:05.536509 7603 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } }
I20260810 18:49:05.536839 7603 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:05.536916 7603 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7da8ab9215743c190c018b4d18715c7, State: Initialized, Role: FOLLOWER
I20260810 18:49:05.537370 7603 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } }
I20260810 18:49:05.537575 7603 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:05.537714 7603 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:05.537886 7603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:05.540091 7603 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } }
I20260810 18:49:05.540514 7603 leader_election.cc:302] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7da8ab9215743c190c018b4d18715c7; no voters:
I20260810 18:49:05.541018 7603 leader_election.cc:288] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:05.541147 7608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:05.541743 7608 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [term 1 LEADER]: Becoming Leader. State: Replica: e7da8ab9215743c190c018b4d18715c7, State: Running, Role: LEADER
I20260810 18:49:05.542330 7608 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } }
I20260810 18:49:05.543258 7603 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:05.545665 7612 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7da8ab9215743c190c018b4d18715c7. Latest consensus state: current_term: 1 leader_uuid: "e7da8ab9215743c190c018b4d18715c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } } }
I20260810 18:49:05.545693 7609 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7da8ab9215743c190c018b4d18715c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7da8ab9215743c190c018b4d18715c7" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39525 } } }
I20260810 18:49:05.546103 7612 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:05.546103 7609 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:05.550925 7618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:05.555768 7618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:05.564082 7618 catalog_manager.cc:1347] Generated new cluster ID: 6d3c0e40110949fa8459589dc3f868f3
I20260810 18:49:05.564173 7618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:05.576694 7618 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:05.578143 7618 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:05.584113 7618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7da8ab9215743c190c018b4d18715c7: Generated new TSK 0
I20260810 18:49:05.585000 7618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:49:05.639941 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:05.955070 7638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:05.955319 7638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:05.955457 7638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:05.962637 7638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:05.962739 7638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:05.962885 7638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:05.972242 7638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:05.973382 7638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:05.974658 7638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:05.986908 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:05.986980 7644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:05.986948 7652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:05.987813 7638 server_base.cc:1034] running on GCE node
I20260810 18:49:05.988368 7638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:05.989185 7638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:05.990401 7638 hybrid_clock.cc:648] HybridClock initialized: now 1786387745990347 us; error 42 us; skew 500 ppm
I20260810 18:49:05.990682 7638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:05 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387745, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:05.993640 7638 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:49:05.995164 7638 webserver.cc:460] Webserver started at http://127.29.174.193:41713/ using document root <none> and password file <none>
I20260810 18:49:05.995821 7638 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:05.995941 7638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:05.996233 7638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:05.999043 7638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "367627489c944a35bd1dedae91bef54c"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "58019991da498fb9ee638bd4b285a094"
server_key_iv: "121409745dac96342c9fe0ce5e17ea1f"
server_key_version: "encryptionkey@0"
I20260810 18:49:05.999795 7638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "367627489c944a35bd1dedae91bef54c"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "58019991da498fb9ee638bd4b285a094"
server_key_iv: "121409745dac96342c9fe0ce5e17ea1f"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.004535 7638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 18:49:06.007740 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.008882 7638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:06.009027 7638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "367627489c944a35bd1dedae91bef54c"
format_stamp: "Formatted at 2026-08-10 18:49:05 on dist-test-slave-6zbq"
server_key: "58019991da498fb9ee638bd4b285a094"
server_key_iv: "121409745dac96342c9fe0ce5e17ea1f"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.009197 7638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:06.022264 7638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:06.026327 7638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:06.026613 7638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:06.027473 7638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:06.028723 7638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:06.028800 7638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.028919 7638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:06.028976 7638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.058619 7638 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:41541
I20260810 18:49:06.058658 7849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:41541 every 8 connection(s)
I20260810 18:49:06.059788 7638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:06.068711 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 7638
I20260810 18:49:06.068902 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:06.069555 30395 external_mini_cluster.cc:1428] Setting key 722bb3bbf063a593c449a1fe98af8abe
Aug 10 18:49:06 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387745, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:06.078519 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.062036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:58809 (local address 127.29.174.254:39525)
0810 18:49:06.062470 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:06.062474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:06.063312 (+ 838us) server_negotiation.cc:407] Connection header received
0810 18:49:06.065016 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:06.065021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:06.065098 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:06.065193 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:06.066633 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.067367 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.069182 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.069593 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.069718 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.073220 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:06.073225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:06.073228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:06.073259 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:06.075692 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:06.076368 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:06.076372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:06.076374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:06.076492 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:06.076890 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:06.076893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:06.076896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:06.077152 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:06.077280 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:06.077911 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 18:49:06.078142 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260810 18:49:06.079486 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.062461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39525 (local address 127.29.174.193:58809)
0810 18:49:06.063162 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:06.063204 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:06.064761 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:06.065424 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:06.065432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:06.065905 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:06.066464 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.066483 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.067520 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.067523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.069022 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.069038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.069818 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.069827 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.070095 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.070720 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:06.070741 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:06.073022 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:06.075828 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:06.075836 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:06.075850 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:06.076246 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:06.076592 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:06.076597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:06.076601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:06.076777 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:06.077271 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:06.077279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:06.077561 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 18:49:06.077772 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 18:49:06.078233 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":276,"threads_started":1}
I20260810 18:49:06.080870 7850 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39525
I20260810 18:49:06.081159 7850 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:06.081789 7850 heartbeater.cc:511] Master 127.29.174.254:39525 requested a full tablet report, sending...
I20260810 18:49:06.083778 7544 ts_manager.cc:194] Registered new tserver with Master: 367627489c944a35bd1dedae91bef54c (127.29.174.193:41541)
I20260810 18:49:06.085278 7544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:58809
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:49:06.121739 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:06.433713 7864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:06.433985 7864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:06.434114 7864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:06.440977 7864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:06.441088 7864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:06.441241 7864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:06.450273 7864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:06.451426 7864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:06.452728 7864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:06.464927 7870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:06.464867 7871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:06.464901 7873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:06.465855 7864 server_base.cc:1034] running on GCE node
I20260810 18:49:06.466429 7864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:06.467221 7864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:06.468410 7864 hybrid_clock.cc:648] HybridClock initialized: now 1786387746468368 us; error 36 us; skew 500 ppm
I20260810 18:49:06.468689 7864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:06 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387746, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:06.471508 7864 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:49:06.472800 7864 webserver.cc:460] Webserver started at http://127.29.174.194:43271/ using document root <none> and password file <none>
I20260810 18:49:06.473383 7864 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:06.473497 7864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:06.473765 7864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:06.476431 7864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "df24a9fccba043a2b15e2dd0290d90d7"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "b831dd705dfb86d414665e59ccd4bdf5"
server_key_iv: "7198f3209544c85584e2b2b1e9a59fca"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.477258 7864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "df24a9fccba043a2b15e2dd0290d90d7"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "b831dd705dfb86d414665e59ccd4bdf5"
server_key_iv: "7198f3209544c85584e2b2b1e9a59fca"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.481894 7864 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:49:06.485025 7882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.486150 7864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:06.486294 7864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "df24a9fccba043a2b15e2dd0290d90d7"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "b831dd705dfb86d414665e59ccd4bdf5"
server_key_iv: "7198f3209544c85584e2b2b1e9a59fca"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.486506 7864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:06.500378 7864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:06.504372 7864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:06.504626 7864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:06.505539 7864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:06.506875 7864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:06.506955 7864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.507113 7864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:06.507215 7864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.536929 7864 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:40351
I20260810 18:49:06.536968 8061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:40351 every 8 connection(s)
I20260810 18:49:06.538053 7864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:06.543324 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 7864
I20260810 18:49:06.543503 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:06.543988 30395 external_mini_cluster.cc:1428] Setting key 921bf75a77d1acfe3e4c7473e6fe97df
Aug 10 18:49:06 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387746, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:06.557041 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.540412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:45599 (local address 127.29.174.254:39525)
0810 18:49:06.540577 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:06.540582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:06.541655 (+ 1073us) server_negotiation.cc:407] Connection header received
0810 18:49:06.543599 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:06.543605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:06.543682 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:06.543779 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:06.545428 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.546160 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.547699 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.548103 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.548210 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.551693 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:06.551698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:06.551701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:06.551731 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:06.554029 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:06.554877 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:06.554883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:06.554887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:06.555052 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:06.555450 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:06.555455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:06.555458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:06.555794 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:06.555941 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:06.556516 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 18:49:06.556729 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:49:06.557722 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:06.540800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39525 (local address 127.29.174.194:45599)
0810 18:49:06.541506 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:06.541542 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:06.543293 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:06.544015 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:06.544023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:06.544646 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:06.545264 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.545282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.546315 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.546319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.547538 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:06.547551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:06.548309 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:06.548318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:06.548531 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:06.549143 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:06.549163 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:06.551498 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:06.554176 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:06.554184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:06.554197 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:06.554725 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:06.555178 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:06.555181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:06.555183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:06.555349 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:06.555896 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:06.555901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:06.556170 (+ 269us) client_negotiation.cc:769] Sending connection context
0810 18:49:06.556364 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:49:06.556768 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260810 18:49:06.559058 8063 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39525
I20260810 18:49:06.559324 8063 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:06.559959 8063 heartbeater.cc:511] Master 127.29.174.254:39525 requested a full tablet report, sending...
I20260810 18:49:06.561242 7544 ts_manager.cc:194] Registered new tserver with Master: df24a9fccba043a2b15e2dd0290d90d7 (127.29.174.194:40351)
I20260810 18:49:06.562002 7544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:45599
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:49:06.591495 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:06.907075 8070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:06.907362 8070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:06.907483 8070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:06.914583 8070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:06.914713 8070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:06.914870 8070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:06.924311 8070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:06.925513 8070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:06.926820 8070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:06.938977 8085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:06.939052 8079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:06.939034 8080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:06.939765 8070 server_base.cc:1034] running on GCE node
I20260810 18:49:06.940301 8070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:06.941164 8070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:06.942375 8070 hybrid_clock.cc:648] HybridClock initialized: now 1786387746942305 us; error 40 us; skew 500 ppm
I20260810 18:49:06.942667 8070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:06 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387746, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:06.945727 8070 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:49:06.947130 8070 webserver.cc:460] Webserver started at http://127.29.174.195:36443/ using document root <none> and password file <none>
I20260810 18:49:06.947781 8070 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:06.947898 8070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:06.948180 8070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:06.951131 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "0eb7c540aac74502824bb9e2b94ad53f"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "06c35d5332d64804a24b1b9243c48f01"
server_key_iv: "20363013ad38ea3785da15723e4d25a0"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.951902 8070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "0eb7c540aac74502824bb9e2b94ad53f"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "06c35d5332d64804a24b1b9243c48f01"
server_key_iv: "20363013ad38ea3785da15723e4d25a0"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.956734 8070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:49:06.959935 8103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:06.961140 8070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 18:49:06.961278 8070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "0eb7c540aac74502824bb9e2b94ad53f"
format_stamp: "Formatted at 2026-08-10 18:49:06 on dist-test-slave-6zbq"
server_key: "06c35d5332d64804a24b1b9243c48f01"
server_key_iv: "20363013ad38ea3785da15723e4d25a0"
server_key_version: "encryptionkey@0"
I20260810 18:49:06.961453 8070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:07.005496 8070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:07.009184 8070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:07.009457 8070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:07.010478 8070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:07.011786 8070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:07.011863 8070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:07.011986 8070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:07.012039 8070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:07.042248 8070 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:44609
I20260810 18:49:07.042297 8267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:44609 every 8 connection(s)
I20260810 18:49:07.043545 8070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:07.050662 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 8070
I20260810 18:49:07.050851 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:07.051286 30395 external_mini_cluster.cc:1428] Setting key 2ce9777918fc622e886131b869eea52b
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387746, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:07.063148 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.046090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:39289 (local address 127.29.174.254:39525)
0810 18:49:07.046319 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.046324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.047491 (+ 1167us) server_negotiation.cc:407] Connection header received
0810 18:49:07.049177 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.049182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.049248 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.049330 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.050913 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.051739 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.053677 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.054215 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.054308 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.057497 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.057502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.057505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.057538 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.060122 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.060983 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.060987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.060990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.061128 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.061556 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.061561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.061563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.061848 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.061957 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.062761 (+ 804us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.062916 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 18:49:07.064057 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.046602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39525 (local address 127.29.174.195:39289)
0810 18:49:07.047345 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:07.047381 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:07.048948 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:07.049503 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:07.049510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:07.049990 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:07.050747 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.050773 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.051889 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.051892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.053547 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:07.053558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.054427 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.054433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:07.054621 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.055259 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:07.055282 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:07.057316 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:07.060281 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:07.060290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:07.060305 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:07.060861 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:07.061235 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:07.061240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:07.061244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:07.061455 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:07.061968 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:07.061975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:07.062412 (+ 437us) client_negotiation.cc:769] Sending connection context
0810 18:49:07.062685 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 18:49:07.063148 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":282,"threads_started":1}
I20260810 18:49:07.065526 8268 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39525
I20260810 18:49:07.065801 8268 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:07.066588 8268 heartbeater.cc:511] Master 127.29.174.254:39525 requested a full tablet report, sending...
I20260810 18:49:07.067847 7542 ts_manager.cc:194] Registered new tserver with Master: 0eb7c540aac74502824bb9e2b94ad53f (127.29.174.195:44609)
I20260810 18:49:07.068674 7542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:39289
I20260810 18:49:07.078037 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:49:07.088140 7850 heartbeater.cc:503] Master 127.29.174.254:39525 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/test-admin.keytab.
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.107203 30395 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 18:49:07.112334 8277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.109686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.29.174.193:41541)
0810 18:49:07.110380 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.110389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.110416 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:49:07.110477 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.110483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.110683 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.110823 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.111864 (+ 1041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47548: BlockingRecv error: recv got EOF from 127.0.0.1:47548 (error 108)
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":342,"threads_started":1}
W20260810 18:49:07.112314 30395 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.113842 30395 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.115104 30395 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:49:07.129673 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.116614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.29.174.193:41541)
0810 18:49:07.116795 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.116800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.116970 (+ 170us) server_negotiation.cc:407] Connection header received
0810 18:49:07.117129 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.117134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.117191 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.117266 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.118185 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.119169 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.120347 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.120885 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.121004 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.124356 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.124367 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.124381 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.124413 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.127101 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.127743 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.127751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.127753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.127871 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.128207 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.128211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.128213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.128560 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.128837 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.129168 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.129427 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 18:49:07.134963 8277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.133386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47556 (local address 127.29.174.193:41541)
0810 18:49:07.133554 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.133560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.133795 (+ 235us) server_negotiation.cc:407] Connection header received
0810 18:49:07.133903 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.133907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.133982 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.134059 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.134760 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47556: BlockingRecv error: recv got EOF from 127.0.0.1:47556 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 18:49:07.135251 30395 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.136557 30395 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.137691 30395 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:49:07.150839 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.138837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.29.174.193:41541)
0810 18:49:07.138995 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.139001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.139290 (+ 289us) server_negotiation.cc:407] Connection header received
0810 18:49:07.139427 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.139432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.139482 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.139553 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.140397 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.141065 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.142234 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.142710 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.142831 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.145965 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.145970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.145973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.146010 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.148489 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.149110 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.149114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.149116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.149237 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.149593 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.149597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.149599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.149861 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.150107 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.150398 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.150588 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 18:49:07.154510 8277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:07.153109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.29.174.193:41541)
0810 18:49:07.153293 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.153298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.153472 (+ 174us) server_negotiation.cc:407] Connection header received
0810 18:49:07.153575 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.153578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.153639 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.153711 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.154283 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47572: BlockingRecv error: recv got EOF from 127.0.0.1:47572 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 18:49:07.154693 30395 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387695213813-30395-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.155993 30395 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:07.157087 30395 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:49:07.169490 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.158199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.29.174.193:41541)
0810 18:49:07.158379 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.158384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.158646 (+ 262us) server_negotiation.cc:407] Connection header received
0810 18:49:07.158785 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.158789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.158837 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.158908 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.159792 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.160466 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.161639 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.162025 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.162128 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.165287 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.165292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.165295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.165321 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.167502 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.168055 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.168058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.168060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.168182 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.168447 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.168450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.168451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.168679 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.168885 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.169107 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.169263 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:49:07.170809 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 7638
I20260810 18:49:07.180825 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.171502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47600 (local address 127.29.174.193:41541)
0810 18:49:07.171629 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.171634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.171865 (+ 231us) server_negotiation.cc:407] Connection header received
0810 18:49:07.171991 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.171994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.172041 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.172110 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.172917 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.173521 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.174641 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.175033 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.175120 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.176745 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.176749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.176752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.176778 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.178976 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.179484 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.179488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.179490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.179605 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.179862 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.179864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.179866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.180074 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.180249 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.180457 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.180593 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:49:07.181365 7843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:07.262465 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 7638
I20260810 18:49:07.282171 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 7864
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:49:07.296823 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.283060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.29.174.194:40351)
0810 18:49:07.283569 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.283576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.283601 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:49:07.283667 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.283674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.283873 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.283993 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.284896 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.285955 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.287285 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.287846 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.287953 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.291639 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.291650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.291663 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.291695 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.294411 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.294981 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.294990 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.294993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.295121 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.295377 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.295380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.295382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.295697 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.295922 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.296286 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.296533 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":257,"threads_started":1}
I20260810 18:49:07.297991 8056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:07.380399 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 7864
I20260810 18:49:07.399719 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 8070
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[7473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:49:07.415428 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:07.401212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.29.174.195:44609)
0810 18:49:07.401929 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:07.401936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:07.401967 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:49:07.402026 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:07.402033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:07.402257 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:07.402439 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:07.403507 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.404550 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.405862 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:07.406625 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:07.406722 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:07.410427 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:07.410438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:07.410452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:07.410486 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:07.413128 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.413674 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.413680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.413683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.413805 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:07.414082 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:07.414085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:07.414087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:07.414449 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:07.414679 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:07.414884 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 18:49:07.415101 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":364,"threads_started":1}
I20260810 18:49:07.416617 8263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:07.499150 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 8070
I20260810 18:49:07.523077 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 7492
I20260810 18:49:07.524113 7595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:07.627553 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 7492
2026-08-10T18:49:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3939 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[8321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[8321](info): set up 2 sockets
Aug 10 18:49:07 dist-test-slave-6zbq krb5kdc[8321](info): commencing operation
krb5kdc: starting...
W20260810 18:49:08.854961 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:08 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab.
Entry for principal oryx/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab.
I20260810 18:49:09.067807 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:37617
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab
--principal=oryx/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:37617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:09.394729 8345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:09.394987 8345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:09.395107 8345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:09.402160 8345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:09.402263 8345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:09.402316 8345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:09.402390 8345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:09.402444 8345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:09.412163 8345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:37617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab
--principal=oryx/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:37617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:09.413388 8345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:09.414654 8345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:09.425827 8351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.425797 8355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.425768 8352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.426577 8345 server_base.cc:1034] running on GCE node
I20260810 18:49:09.427311 8345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:09.428579 8345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:09.429769 8345 hybrid_clock.cc:648] HybridClock initialized: now 1786387749429722 us; error 38 us; skew 500 ppm
I20260810 18:49:09.430088 8345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:09 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:09.433050 8345 init.cc:377] Logged in from keytab as oryx/127.29.174.254@KRBTEST.COM (short username oryx)
I20260810 18:49:09.434326 8345 webserver.cc:460] Webserver started at http://127.29.174.254:34975/ using document root <none> and password file <none>
I20260810 18:49:09.434958 8345 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:09.435055 8345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:09.435298 8345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:09.438032 8345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "1f5fcde8ba364546992fca787dbc4e05"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "d9b28ce0143d6b986b31ea97cb27e925"
server_key_iv: "2209b8427cfa664af5d38d3b29eab144"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.438776 8345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "1f5fcde8ba364546992fca787dbc4e05"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "d9b28ce0143d6b986b31ea97cb27e925"
server_key_iv: "2209b8427cfa664af5d38d3b29eab144"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.443425 8345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:49:09.446584 8364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:09.447789 8345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:49:09.447937 8345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "1f5fcde8ba364546992fca787dbc4e05"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "d9b28ce0143d6b986b31ea97cb27e925"
server_key_iv: "2209b8427cfa664af5d38d3b29eab144"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.448098 8345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:09.460505 8345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:09.464289 8345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:09.464531 8345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:09.480921 8345 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:37617
I20260810 18:49:09.480952 8448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:37617 every 8 connection(s)
I20260810 18:49:09.482149 8345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:09.485437 8449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:09.487929 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 8345
I20260810 18:49:09.488112 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:09.488752 30395 external_mini_cluster.cc:1428] Setting key f398a6ca3e1741b2411bc0bde10dc30f
I20260810 18:49:09.497605 8449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Bootstrap starting.
I20260810 18:49:09.501061 8449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:09.502308 8449 log.cc:824] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:49:09 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:09.505329 8449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: No bootstrap required, opened a new log
I20260810 18:49:09.511732 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:09.490665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.29.174.254:37617)
0810 18:49:09.491451 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:09.491467 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:09.491517 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:49:09.493605 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:09.493633 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:09.494103 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:09.494541 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:09.495805 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.497659 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.498912 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:09.499918 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:09.500104 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:09.505031 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:09.505039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:09.505052 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:09.505089 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:09.507927 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:09.508692 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:09.508697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:09.508706 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:09.508859 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:09.509220 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:09.509223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:09.509225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:09.509734 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:09.509971 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:09.510459 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 18:49:09.510808 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":314,"threads_started":1}
I20260810 18:49:09.513479 8449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } }
I20260810 18:49:09.513803 8449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:09.513902 8449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5fcde8ba364546992fca787dbc4e05, State: Initialized, Role: FOLLOWER
I20260810 18:49:09.514472 8449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } }
I20260810 18:49:09.514694 8449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:09.514818 8449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:09.514961 8449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:09.517133 8449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } }
I20260810 18:49:09.517628 8449 leader_election.cc:302] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f5fcde8ba364546992fca787dbc4e05; no voters:
I20260810 18:49:09.518148 8449 leader_election.cc:288] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:09.518258 8456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:09.518921 8456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 LEADER]: Becoming Leader. State: Replica: 1f5fcde8ba364546992fca787dbc4e05, State: Running, Role: LEADER
I20260810 18:49:09.519403 8456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } }
I20260810 18:49:09.520012 8449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:09.521952 8464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f5fcde8ba364546992fca787dbc4e05. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } }
I20260810 18:49:09.522324 8464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:09.521919 8461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } }
I20260810 18:49:09.522874 8461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:09.524827 8472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:09.529975 8472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:09.538692 8472 catalog_manager.cc:1347] Generated new cluster ID: f9df7d5a26814eff8825735d74bc0e0f
I20260810 18:49:09.538784 8472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:09.576254 8472 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:09.577580 8472 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:09.583077 8472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Generated new TSK 0
I20260810 18:49:09.583825 8472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
I20260810 18:49:09.640074 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:09.954841 8489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:09.955125 8489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:09.955258 8489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:09.962404 8489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:09.962513 8489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:09.962667 8489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:09.972115 8489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:09.973304 8489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:09.974563 8489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:09.986696 8497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.986652 8498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:09.986850 8501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:09.987591 8489 server_base.cc:1034] running on GCE node
I20260810 18:49:09.988107 8489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:09.988943 8489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:09.990137 8489 hybrid_clock.cc:648] HybridClock initialized: now 1786387749990093 us; error 41 us; skew 500 ppm
I20260810 18:49:09.990468 8489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:09 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:09.993353 8489 init.cc:377] Logged in from keytab as oryx/127.29.174.193@KRBTEST.COM (short username oryx)
I20260810 18:49:09.994637 8489 webserver.cc:460] Webserver started at http://127.29.174.193:44137/ using document root <none> and password file <none>
I20260810 18:49:09.995232 8489 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:09.995342 8489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:09.995608 8489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:09.998332 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "edef890e214b2fca8d221f60277ecbb8"
server_key_iv: "2b12b092fd51806db1c46116deae520f"
server_key_version: "encryptionkey@0"
I20260810 18:49:09.999110 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "edef890e214b2fca8d221f60277ecbb8"
server_key_iv: "2b12b092fd51806db1c46116deae520f"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.003760 8489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:10.006788 8516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.007946 8489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:10.008083 8489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "edef890e214b2fca8d221f60277ecbb8"
server_key_iv: "2b12b092fd51806db1c46116deae520f"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.008256 8489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.024457 8489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.028178 8489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.028425 8489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.029277 8489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.030527 8489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.030597 8489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.030746 8489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.030798 8489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.059000 8489 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:39495
I20260810 18:49:10.059047 8681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:39495 every 8 connection(s)
I20260810 18:49:10.060130 8489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:10.070554 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 8489
I20260810 18:49:10.070753 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:10.071379 30395 external_mini_cluster.cc:1428] Setting key c7c5a3240b6105e0a708354a0d54e192
Aug 10 18:49:10 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:10.078231 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.062395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:41221 (local address 127.29.174.254:37617)
0810 18:49:10.062845 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.062851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.063714 (+ 863us) server_negotiation.cc:407] Connection header received
0810 18:49:10.065362 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.065367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.065433 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.065526 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.066910 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.067674 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.069148 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.069556 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.069663 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.073004 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.073011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.073015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:10.073054 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.075499 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:10.076265 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:10.076269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:10.076271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:10.076376 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:10.076769 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:10.076772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:10.076774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:10.077021 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.077124 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.077810 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.077974 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":203,"threads_started":1}
I20260810 18:49:10.079447 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.062874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:41221)
0810 18:49:10.063562 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.063602 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.065127 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.065699 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.065706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.066150 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.066735 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.066752 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.067821 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.067825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.068993 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.069005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.069727 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.069732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.069921 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.070583 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:10.070603 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.072815 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.075646 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:10.075653 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:10.075666 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:10.076162 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:10.076460 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:10.076463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:10.076467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:10.076679 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:10.077129 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.077135 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.077491 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.077760 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.078213 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":269,"threads_started":1}
I20260810 18:49:10.080804 8682 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:10.081082 8682 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.081755 8682 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:10.083823 8392 ts_manager.cc:194] Registered new tserver with Master: 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:10.085420 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:41221
WARNING: no policy specified for oryx/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
I20260810 18:49:10.121327 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:10.433975 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.434237 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.434374 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.441342 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.441448 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.441592 8691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:10.450578 8691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:10.451691 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.452978 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.464846 8701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.465116 8706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.465157 8700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.465755 8691 server_base.cc:1034] running on GCE node
I20260810 18:49:10.466320 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.467181 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.468372 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786387750468332 us; error 31 us; skew 500 ppm
I20260810 18:49:10.468658 8691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:10 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387750, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:10.475795 8691 init.cc:377] Logged in from keytab as oryx/127.29.174.194@KRBTEST.COM (short username oryx)
I20260810 18:49:10.477056 8691 webserver.cc:460] Webserver started at http://127.29.174.194:42201/ using document root <none> and password file <none>
I20260810 18:49:10.477670 8691 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.477787 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.478055 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.480849 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "47c487ac956942a191e7b1aabc41d3da"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "417819400e78abf93114a9dc7897b1f3"
server_key_iv: "46edee3e5440d28ce7dd3712cdc334cb"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.481601 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "47c487ac956942a191e7b1aabc41d3da"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "417819400e78abf93114a9dc7897b1f3"
server_key_iv: "46edee3e5440d28ce7dd3712cdc334cb"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.486788 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 18:49:10.490854 8715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.492931 8691 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:10.493110 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "47c487ac956942a191e7b1aabc41d3da"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "417819400e78abf93114a9dc7897b1f3"
server_key_iv: "46edee3e5440d28ce7dd3712cdc334cb"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.493340 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.518440 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.522392 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.522642 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.523521 8691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.524750 8691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.524821 8691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.524937 8691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.524987 8691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.554555 8691 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:32895
I20260810 18:49:10.554618 8895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:32895 every 8 connection(s)
I20260810 18:49:10.555706 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:10.560765 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 8691
I20260810 18:49:10.560935 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:10.561388 30395 external_mini_cluster.cc:1428] Setting key 6b52336a245281d31b3e83f652bd9bd9
Aug 10 18:49:10 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387750, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:10.574887 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.558203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:50381 (local address 127.29.174.254:37617)
0810 18:49:10.558403 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:10.558409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:10.559506 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 18:49:10.561342 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:10.561348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:10.561436 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:10.561531 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:10.563375 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.564078 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.565528 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.565926 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.566039 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.569474 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:10.569479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:10.569483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:10.569514 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:10.571883 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:10.572535 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:10.572539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:10.572541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:10.572667 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:10.573134 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:10.573138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:10.573141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:10.573395 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:10.573508 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:10.574321 (+ 813us) server_negotiation.cc:299] Negotiation successful
0810 18:49:10.574573 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:49:10.575695 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:10.558616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:50381)
0810 18:49:10.559358 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:10.559402 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:10.561088 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:10.561905 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:10.561915 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:10.562480 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:10.563213 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.563233 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.564226 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.564231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.565392 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:10.565405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:10.566135 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:10.566142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:10.566385 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:10.566972 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:10.566992 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:10.569279 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:10.572010 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:10.572018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:10.572032 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:10.572416 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:10.572775 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:10.572780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:10.572784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:10.573008 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:10.573515 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:10.573523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:10.573894 (+ 371us) client_negotiation.cc:769] Sending connection context
0810 18:49:10.574168 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 18:49:10.574638 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":285,"threads_started":1}
I20260810 18:49:10.576982 8899 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:10.577226 8899 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:10.577859 8899 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:10.579259 8392 ts_manager.cc:194] Registered new tserver with Master: 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:10.580232 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:50381
WARNING: no policy specified for oryx/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
I20260810 18:49:10.612727 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:10.918686 8906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:10.918953 8906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:10.919071 8906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:10.926676 8906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:10.926793 8906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:10.926942 8906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:10.936127 8906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:37617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:10.937258 8906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:10.938481 8906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:10.950556 8921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.950515 8924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:10.950619 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:10.951356 8906 server_base.cc:1034] running on GCE node
I20260810 18:49:10.951910 8906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:10.952834 8906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:10.954023 8906 hybrid_clock.cc:648] HybridClock initialized: now 1786387750953977 us; error 43 us; skew 500 ppm
I20260810 18:49:10.954300 8906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:10 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387750, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:10.957108 8906 init.cc:377] Logged in from keytab as oryx/127.29.174.195@KRBTEST.COM (short username oryx)
I20260810 18:49:10.958393 8906 webserver.cc:460] Webserver started at http://127.29.174.195:43791/ using document root <none> and password file <none>
I20260810 18:49:10.958972 8906 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:10.959087 8906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:10.959363 8906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:10.962085 8906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "69e1e6de0bd9433aac8cde7a713a6337"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "9ff8ab9ddd21e93efff7e4513c39d828"
server_key_iv: "65cecf16b7c221d7374a057963588dbd"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.962855 8906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "69e1e6de0bd9433aac8cde7a713a6337"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "9ff8ab9ddd21e93efff7e4513c39d828"
server_key_iv: "65cecf16b7c221d7374a057963588dbd"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.967446 8906 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:49:10.970678 8933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.971869 8906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:10.972015 8906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "69e1e6de0bd9433aac8cde7a713a6337"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "9ff8ab9ddd21e93efff7e4513c39d828"
server_key_iv: "65cecf16b7c221d7374a057963588dbd"
server_key_version: "encryptionkey@0"
I20260810 18:49:10.972177 8906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:10.990653 8906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:10.994134 8906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:10.994437 8906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:10.995302 8906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:10.996577 8906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:10.996650 8906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:10.996766 8906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:10.996819 8906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.025533 8906 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34889
I20260810 18:49:11.025568 9114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34889 every 8 connection(s)
I20260810 18:49:11.026746 8906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:11.030771 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 8906
I20260810 18:49:11.030920 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:11.031345 30395 external_mini_cluster.cc:1428] Setting key b5d281b7f70bc314d5ddce7b1613f202
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387750, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:11.044018 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.028880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:57913 (local address 127.29.174.254:37617)
0810 18:49:11.029079 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.029086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.030112 (+ 1026us) server_negotiation.cc:407] Connection header received
0810 18:49:11.031858 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.031862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.031931 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.032022 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.033550 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.034314 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.035767 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.036178 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.036283 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.039340 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.039345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.039349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.039380 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.041640 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.042260 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.042263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.042266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.042414 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.042758 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.042761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.042763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.042997 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.043111 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.043631 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.043787 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:49:11.044904 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.029316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:57913)
0810 18:49:11.029970 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.030003 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.031633 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.032273 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.032281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.032834 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.033391 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.033408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.034471 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.034475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.035625 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.035637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.036396 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.036403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.036610 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.037117 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.037142 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.039157 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.041772 (+ 2615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.041779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.041789 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.042174 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.042499 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.042502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.042505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.042675 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.043088 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.043094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.043363 (+ 269us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.043586 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.043906 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":256,"threads_started":1}
I20260810 18:49:11.046097 9116 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:11.046404 9116 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:11.047043 9116 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:11.048288 8392 ts_manager.cc:194] Registered new tserver with Master: 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889)
I20260810 18:49:11.049052 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:57913
I20260810 18:49:11.057989 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.174.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.253@KRBTEST.COM" created.
I20260810 18:49:11.088101 8682 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.253.keytab.
Entry for principal oryx/127.29.174.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.253.keytab.
I20260810 18:49:11.109794 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:40749
--webserver_interface=127.29.174.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.253.keytab
--principal=oryx/127.29.174.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:37617,127.29.174.253:40749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:11.420974 9124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:11.421264 9124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:11.421394 9124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:11.428488 9124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:11.428591 9124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:11.428668 9124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:11.428745 9124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:11.428805 9124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:11.438196 9124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:37617,127.29.174.253:40749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.253.keytab
--principal=oryx/127.29.174.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:40749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.174.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
W20260810 18:49:11.438534 9124 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.174.254:37617,127.29.174.253:40749'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 18:49:11.439424 9124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:11.440603 9124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:11.451269 9131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.451225 9135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.451308 9132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:11.452309 9124 server_base.cc:1034] running on GCE node
I20260810 18:49:11.452998 9124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:11.454228 9124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:11.455433 9124 hybrid_clock.cc:648] HybridClock initialized: now 1786387751455377 us; error 58 us; skew 500 ppm
I20260810 18:49:11.455713 9124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387751, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:11.458555 9124 init.cc:377] Logged in from keytab as oryx/127.29.174.253@KRBTEST.COM (short username oryx)
I20260810 18:49:11.459838 9124 webserver.cc:460] Webserver started at http://127.29.174.253:42531/ using document root <none> and password file <none>
I20260810 18:49:11.460448 9124 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:11.460543 9124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:11.460825 9124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:11.463620 9124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/instance:
uuid: "54c26ffbaced4013a95c6a6a108cce52"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-6zbq"
server_key: "1c17f8fca67f63c69319f7649bf1d7fb"
server_key_iv: "1ca2dc16f4c08e63645564ed10546b9f"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.464342 9124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal/instance:
uuid: "54c26ffbaced4013a95c6a6a108cce52"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-6zbq"
server_key: "1c17f8fca67f63c69319f7649bf1d7fb"
server_key_iv: "1ca2dc16f4c08e63645564ed10546b9f"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.469084 9124 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:49:11.472292 9146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.473426 9124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:49:11.473574 9124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "54c26ffbaced4013a95c6a6a108cce52"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-6zbq"
server_key: "1c17f8fca67f63c69319f7649bf1d7fb"
server_key_iv: "1ca2dc16f4c08e63645564ed10546b9f"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.473757 9124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:11.518004 9124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:11.521986 9124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:11.522223 9124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:11.540004 9124 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.253:40749
I20260810 18:49:11.540009 9226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.253:40749 every 8 connection(s)
I20260810 18:49:11.541288 9124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
I20260810 18:49:11.544783 9227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:11.547222 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 9124
I20260810 18:49:11.551956 9227 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387751, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.253@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:11.570657 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.554640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.29.174.254:37617)
0810 18:49:11.555164 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.555170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.555742 (+ 572us) server_negotiation.cc:407] Connection header received
0810 18:49:11.555817 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.555822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.555902 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.556012 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.557506 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.558749 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.560498 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.561040 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.561193 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.564598 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.564605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.564610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.564648 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.567560 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.568681 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.568687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.568690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.568823 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.569173 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.569177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.569179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.569479 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.569612 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.570093 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.570290 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":257,"threads_started":1}
I20260810 18:49:11.571293 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.554649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56208)
0810 18:49:11.555469 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.555518 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.555668 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.556086 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.556093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.556983 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.557345 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.557358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.558957 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.558963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.560346 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.560359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.561258 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.561263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.561381 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.561924 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.561943 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.564412 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.568156 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.568164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.568169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.568566 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.568928 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.568933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.568936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.569063 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.569605 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.569612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.569796 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.569974 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.570454 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":132,"thread_start_us":455,"threads_started":1}
I20260810 18:49:11.571316 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.549300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.29.174.253:40749)
0810 18:49:11.550031 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.550043 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.550080 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:49:11.551816 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.551845 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.552230 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.552534 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.555557 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.557106 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.558928 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.559865 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.559970 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.564271 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.564279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.564289 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.564324 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.567538 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.568139 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.568145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.568150 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.568309 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.568689 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.568693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.568695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.569152 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.569422 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.569831 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.570283 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":535,"thread_start_us":328,"threads_started":1}
I20260810 18:49:11.573066 9227 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387751, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.253@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
I20260810 18:49:11.582475 8899 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
I20260810 18:49:11.584388 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.573806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37640)
0810 18:49:11.574070 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.574083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.574161 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.574414 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.574417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.574693 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.574813 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.574820 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.575735 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.575738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.576605 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.576616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.577283 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.577287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.577417 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.577869 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.577884 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.579742 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.582265 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.582269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.582272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.582708 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.583063 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.583066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.583069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.583227 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.583712 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.583717 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.583856 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.583980 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.584170 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260810 18:49:11.584633 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.573915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.29.174.253:40749)
0810 18:49:11.574052 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.574056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.574122 (+ 66us) server_negotiation.cc:407] Connection header received
0810 18:49:11.574232 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.574235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.574283 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.574372 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.574919 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.575617 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.576722 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.577157 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.577249 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.579900 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.579932 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.579936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.579964 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.582086 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.582819 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.582823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.582825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.582955 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.583352 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.583355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.583357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.583596 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.583703 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.584111 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.584318 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:49:11.588646 9227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Bootstrap starting.
I20260810 18:49:11.591414 9227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:11.592334 9227 log.cc:824] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:11.594813 9227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: No bootstrap required, opened a new log
I20260810 18:49:11.601547 9227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } }
I20260810 18:49:11.601822 9227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:11.601897 9227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c26ffbaced4013a95c6a6a108cce52, State: Initialized, Role: FOLLOWER
I20260810 18:49:11.602373 9227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } }
I20260810 18:49:11.603091 9237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } } }
I20260810 18:49:11.603456 9237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:11.603998 9227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:49:11.612116 9250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:49:11.612251 9250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:49:11.615535 9124 master_runner.cc:428] Detected that this master 54c26ffbaced4013a95c6a6a108cce52 is joining an existing cluster
I20260810 18:49:11.615630 9124 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.174.253:40749
I20260810 18:49:11.635106 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.620024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.29.174.254:37617)
0810 18:49:11.620201 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.620207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.620316 (+ 109us) server_negotiation.cc:407] Connection header received
0810 18:49:11.620587 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.620591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.620681 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.620752 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.622388 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.623446 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.625749 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.626418 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.626514 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.628944 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.628950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.628954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.628987 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.632166 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.632899 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.632905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.632908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.633049 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.633493 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.633498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.633501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.633836 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.633952 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.634585 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.634809 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:49:11.635316 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.619926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.29.174.253:40749)
0810 18:49:11.620198 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.620203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.620811 (+ 608us) server_negotiation.cc:407] Connection header received
0810 18:49:11.620939 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.620944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.621011 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.621106 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.622254 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.623385 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.625491 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.626197 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.626296 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.628819 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.628826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.628831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.628863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.632276 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.633147 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.633152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.633155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.633308 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.633716 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.633721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.633724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.634057 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.634213 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.634739 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.634948 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:49:11.636299 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.619521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37646)
0810 18:49:11.620718 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.620731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.620845 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.621160 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.621164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.621784 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.622114 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.622123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.623584 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.623590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.625333 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.625348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.626597 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.626603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.626770 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.627396 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.627420 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.628651 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.632419 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.632424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.632428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.633012 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.633435 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.633440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.633442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.633604 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.634309 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.634313 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.634484 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.635526 (+ 1042us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.635788 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":24,"thread_start_us":922,"threads_started":1}
I20260810 18:49:11.636404 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.619468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56220)
0810 18:49:11.620221 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.620239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.620433 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.620988 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.620997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.621818 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.622219 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.622233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.623597 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.623602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.625583 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.625597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.626846 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.626852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.627021 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.627533 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.627554 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.628751 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.632332 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.632337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.632340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.632777 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.633183 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.633187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.633190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.633365 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.634039 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.634043 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.634230 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.635679 (+ 1449us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.635969 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":321,"thread_start_us":596,"threads_started":1}
WARNING: no policy specified for oryx/127.29.174.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.252@KRBTEST.COM" created.
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.254@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
I20260810 18:49:11.662956 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.643026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.29.174.253:40749)
0810 18:49:11.643162 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.643168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.646954 (+ 3786us) server_negotiation.cc:407] Connection header received
0810 18:49:11.647281 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.647286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.647376 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.647476 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.648787 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.649873 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.651740 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.652529 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.652653 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.656401 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.656407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.656412 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.656446 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.659776 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.660668 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.660673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.660676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.660845 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.661334 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.661339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.661342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.661654 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.661810 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.662401 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.662610 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:49:11.662983 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.642996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37662)
0810 18:49:11.646838 (+ 3842us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.646874 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.647082 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.647548 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.647555 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.648344 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.648619 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.648632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.650024 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.650030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.651572 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.651588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.652682 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.652687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.652828 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.653381 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.653402 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.656202 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.659923 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.659936 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.659943 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.660521 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.660965 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.660971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.660974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.661202 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.661796 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.661804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.661996 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.662217 (+ 221us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.662577 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3616,"thread_start_us":3479,"threads_started":1}
I20260810 18:49:11.664659 8392 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.174.253:40749: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52"
member_type: NON_VOTER
last_known_addr {
host: "127.29.174.253"
port: 40749
}
attrs {
promote: true
}
}
dest_uuid: "1f5fcde8ba364546992fca787dbc4e05"
cas_config_opid_index: -1
I20260810 18:49:11.665889 8392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } }
I20260810 18:49:11.669513 9193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:11.673219 9193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 18:49:11.678591 9277 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:11.678654 9278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.683508 9278 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.174.253:40749
I20260810 18:49:11.684166 9277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.684517 9277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:11.684721 9124 master.cc:537] Master@127.29.174.253:40749 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.252.keytab.
Entry for principal oryx/127.29.174.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.252.keytab.
I20260810 18:49:11.694742 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:40381
--webserver_interface=127.29.174.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.252.keytab
--principal=oryx/127.29.174.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:49:11.698680 9193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.702728 9283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.702960 9283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:11.702917 9237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f5fcde8ba364546992fca787dbc4e05. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.703104 9237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:11.703382 9287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.703677 9287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:11.720525 9124 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Raft consensus shutting down.
I20260810 18:49:11.720925 9124 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Raft consensus is shut down!
I20260810 18:49:11.721086 9124 tablet_replica.cc:330] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: stopping tablet replica
I20260810 18:49:11.727845 9124 master.cc:552] Master@127.29.174.253:40749 shutdown complete.
I20260810 18:49:11.727964 9124 master_runner.cc:305] Clearing existing system tablet
I20260810 18:49:11.728353 9124 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:11.728438 9124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:11.730513 9124 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 18:49:11.731865 9292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.732164 9124 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:11.732450 9124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:49:11.732532 9124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "54c26ffbaced4013a95c6a6a108cce52"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-6zbq"
server_key: "1c17f8fca67f63c69319f7649bf1d7fb"
server_key_iv: "1ca2dc16f4c08e63645564ed10546b9f"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.735847 9124 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:11.740114 9124 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:49:11.740276 9124 log.cc:1196] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 18:49:11.740646 9124 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Deleting consensus metadata
I20260810 18:49:11.740854 9124 master_runner.cc:315] Copying system tablet from 127.29.174.254:37617
I20260810 18:49:11.743322 9124 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: tablet copy: Beginning tablet copy session from remote peer at address 127.29.174.254:37617
I20260810 18:49:11.755518 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.744169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.29.174.254:37617)
0810 18:49:11.744360 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:11.744367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:11.744697 (+ 330us) server_negotiation.cc:407] Connection header received
0810 18:49:11.744878 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:11.744883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:11.744955 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:11.745040 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:11.745882 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.746814 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.748036 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.748475 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.748584 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.750289 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:11.750294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:11.750297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:11.750330 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:11.752985 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.753593 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.753599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.753602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.753743 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:11.754091 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:11.754096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:11.754100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:11.754439 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:11.754569 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:11.754964 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 18:49:11.755171 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:49:11.755563 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:11.744122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56236)
0810 18:49:11.744593 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:11.744612 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:11.744749 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:11.745084 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:11.745090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:11.745493 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:11.745735 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.745750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.746964 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.746969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.747892 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:11.747905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:11.748622 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:11.748628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:11.748812 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:11.749377 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:11.749394 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:11.750094 (+ 700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:11.753106 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.753111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.753116 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.753474 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.753851 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:11.753856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:11.753860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:11.753989 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:11.754551 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:11.754555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:11.754711 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 18:49:11.754829 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:49:11.755139 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":239,"threads_started":1}
I20260810 18:49:11.756665 8431 tablet_copy_service.cc:140] P 1f5fcde8ba364546992fca787dbc4e05: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 54c26ffbaced4013a95c6a6a108cce52 ({username='oryx', principal='oryx/127.29.174.253@KRBTEST.COM'} at 127.0.0.1:56236)
I20260810 18:49:11.757035 8431 tablet_copy_service.cc:161] P 1f5fcde8ba364546992fca787dbc4e05: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 54c26ffbaced4013a95c6a6a108cce52 at {username='oryx', principal='oryx/127.29.174.253@KRBTEST.COM'} at 127.0.0.1:56236: session id = 54c26ffbaced4013a95c6a6a108cce52-00000000000000000000000000000000
I20260810 18:49:11.760331 8431 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:49:11.762444 9124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:49:11.767088 9124 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: tablet copy: Starting download of 0 data blocks...
I20260810 18:49:11.767422 9124 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: tablet copy: Starting download of 1 WAL segments...
I20260810 18:49:11.770391 9124 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:49:11.773496 8431 tablet_copy_service.cc:342] P 1f5fcde8ba364546992fca787dbc4e05: Request end of tablet copy session 54c26ffbaced4013a95c6a6a108cce52-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.174.253@KRBTEST.COM'} at 127.0.0.1:56236
I20260810 18:49:11.773697 8431 tablet_copy_service.cc:433] P 1f5fcde8ba364546992fca787dbc4e05: ending tablet copy session 54c26ffbaced4013a95c6a6a108cce52-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 54c26ffbaced4013a95c6a6a108cce52
I20260810 18:49:11.783496 9124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:11.786239 9305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.796083 9310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:11.797305 9306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:11.798636 9124 server_base.cc:1034] running on GCE node
I20260810 18:49:11.798995 9124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:11.799574 9124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:11.800841 9124 hybrid_clock.cc:648] HybridClock initialized: now 1786387751800784 us; error 35 us; skew 500 ppm
I20260810 18:49:11.801047 9124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:11 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387751, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:11.802778 9124 init.cc:377] Logged in from keytab as oryx/127.29.174.253@KRBTEST.COM (short username oryx)
I20260810 18:49:11.803880 9124 webserver.cc:460] Webserver started at http://127.29.174.253:32965/ using document root <none> and password file <none>
I20260810 18:49:11.804205 9124 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:11.804280 9124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:11.806550 9124 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:49:11.808207 9317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:11.808640 9124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:49:11.808753 9124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "54c26ffbaced4013a95c6a6a108cce52"
format_stamp: "Formatted at 2026-08-10 18:49:11 on dist-test-slave-6zbq"
server_key: "1c17f8fca67f63c69319f7649bf1d7fb"
server_key_iv: "1ca2dc16f4c08e63645564ed10546b9f"
server_key_version: "encryptionkey@0"
I20260810 18:49:11.808897 9124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:11.822288 9124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:11.825728 9124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:11.851670 9124 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.253:40749
I20260810 18:49:11.852509 9124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
I20260810 18:49:11.851636 9404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.253:40749 every 8 connection(s)
I20260810 18:49:11.858247 9407 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:49:11.861238 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Bootstrap starting.
I20260810 18:49:11.872596 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:49:11.873076 9407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: Bootstrap complete.
I20260810 18:49:11.873922 9407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } }
I20260810 18:49:11.874115 9407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54c26ffbaced4013a95c6a6a108cce52, State: Initialized, Role: LEARNER
I20260810 18:49:11.874411 9407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } }
I20260810 18:49:11.875236 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:11.875548 9407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:11.875712 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:11.882822 9425 catalog_manager.cc:1259] Loaded cluster ID: f9df7d5a26814eff8825735d74bc0e0f
I20260810 18:49:11.882894 9425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: loading cluster ID for follower catalog manager: success
I20260810 18:49:11.885221 9425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: acquiring CA information for follower catalog manager: success
I20260810 18:49:11.887073 9425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:49:12.051234 9116 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
W20260810 18:49:12.158041 9289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:12.158383 9289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:12.158514 9289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:12.168251 9289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:12.168357 9289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:12.168404 9289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:12.168460 9289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:12.168504 9289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:12.181185 9289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.252.keytab
--principal=oryx/127.29.174.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.174.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:12.182487 9289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:12.183921 9289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:12.198206 9432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.199227 9433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.199430 9436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.200315 9289 server_base.cc:1034] running on GCE node
I20260810 18:49:12.201292 9289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:12.202865 9289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:12.204108 9289 hybrid_clock.cc:648] HybridClock initialized: now 1786387752204011 us; error 38 us; skew 500 ppm
I20260810 18:49:12.204541 9289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:12.208531 9289 init.cc:377] Logged in from keytab as oryx/127.29.174.252@KRBTEST.COM (short username oryx)
I20260810 18:49:12.210330 9289 webserver.cc:460] Webserver started at http://127.29.174.252:40965/ using document root <none> and password file <none>
I20260810 18:49:12.211193 9289 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.211390 9289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.211750 9289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:12.215459 9289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/instance:
uuid: "defc7503518a41b0a103c6be9fc8b4ec"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-6zbq"
server_key: "d40b546dc7b8be2e6b3b9b500c30587f"
server_key_iv: "f763b0cb1ca9fa1f37b13e36229b5581"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.216367 9289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal/instance:
uuid: "defc7503518a41b0a103c6be9fc8b4ec"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-6zbq"
server_key: "d40b546dc7b8be2e6b3b9b500c30587f"
server_key_iv: "f763b0cb1ca9fa1f37b13e36229b5581"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.222455 9289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:49:12.226928 9449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.228196 9289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:49:12.228353 9289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "defc7503518a41b0a103c6be9fc8b4ec"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-6zbq"
server_key: "d40b546dc7b8be2e6b3b9b500c30587f"
server_key_iv: "f763b0cb1ca9fa1f37b13e36229b5581"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.228564 9289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:12.244402 9289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:12.251539 9289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:12.252095 9289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:12.287174 9289 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.252:40381
I20260810 18:49:12.287205 9524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.252:40381 every 8 connection(s)
I20260810 18:49:12.288688 9289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
I20260810 18:49:12.293548 9525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:12.293835 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 9289
I20260810 18:49:12.316147 9527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:12.316075 9525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:49:12.317406 9527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
I20260810 18:49:12.339931 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.298389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.29.174.253:40749)
0810 18:49:12.327123 (+ 28734us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.327130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.327152 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:49:12.327231 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.327238 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.327331 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.327454 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:12.328996 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.331007 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.335551 (+ 4544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.337329 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.337633 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.337864 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.339269 (+ 1405us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.339456 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28575,"thread_start_us":34808,"threads_started":1}
I20260810 18:49:12.340240 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.297727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37678)
0810 18:49:12.324839 (+ 27112us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.324862 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.325060 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.327962 (+ 2902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.327968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.328311 (+ 343us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:12.328741 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.328757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.331230 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.331236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.335280 (+ 4044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.335300 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.338047 (+ 2747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.338055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.338312 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.338412 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.338511 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.338676 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26965,"thread_start_us":27426,"threads_started":1}
I20260810 18:49:12.342394 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f5fcde8ba364546992fca787dbc4e05. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: NON_VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: true } } }
I20260810 18:49:12.342769 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:12.343658 9532 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: attempting to promote NON_VOTER 54c26ffbaced4013a95c6a6a108cce52 to VOTER
I20260810 18:49:12.344648 9532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } }
I20260810 18:49:12.347179 9370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 LEARNER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:49:12.348531 9532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 18:49:12.351312 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.296409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.29.174.252:40381)
0810 18:49:12.311905 (+ 15496us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.311925 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.317980 (+ 6055us) server_negotiation.cc:407] Connection header received
0810 18:49:12.321944 (+ 3964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.321978 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.322620 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.325673 (+ 3053us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.326308 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.328725 (+ 2417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.331331 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.332629 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.332797 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.340363 (+ 7566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.340374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.340395 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.340470 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.345463 (+ 4993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.346145 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.346153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.346161 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.346389 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.347086 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.347092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.347095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.348040 (+ 945us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.348506 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.348936 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.349367 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":15175,"thread_start_us":15413,"threads_started":1}
I20260810 18:49:12.352526 9537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } }
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.252@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:12.355808 9370 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } }
I20260810 18:49:12.357220 9537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } }
I20260810 18:49:12.357579 9537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:12.357916 9537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } }
I20260810 18:49:12.358244 9537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:12.361950 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } }
I20260810 18:49:12.362250 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:12.364882 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.320607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.29.174.254:37617)
0810 18:49:12.341950 (+ 21343us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.341957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.341984 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:49:12.342042 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.342046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.342123 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.342244 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.344917 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.346655 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.348829 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.349602 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.349711 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.354244 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.354250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.354253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.354288 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.361108 (+ 6820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.362305 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.362311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.362315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.362572 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.363054 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.363058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.363060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.363446 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.363594 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.364282 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.364470 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21017,"thread_start_us":20754,"threads_started":1}
I20260810 18:49:12.364935 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.320729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56240)
0810 18:49:12.334244 (+ 13515us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.334282 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.334450 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.342522 (+ 8072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.342530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.343887 (+ 1357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.344691 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.344709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.346776 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.346781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.348667 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.348682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.349864 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.349872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.350125 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.350888 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.350922 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.354045 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.361314 (+ 7269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.361330 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.361338 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.362136 (+ 798us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.362708 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.362711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.362713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.362911 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.363621 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.363628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.363860 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.364055 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.364431 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13240,"mutex_wait_us":28,"thread_start_us":13027,"threads_started":1}
I20260810 18:49:12.366396 9525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.252@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
I20260810 18:49:12.381399 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.367304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37692)
0810 18:49:12.367476 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.367495 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.367593 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.367845 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.367848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.368148 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.368334 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.368343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.369375 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.369379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.370559 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.370572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.371291 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.371296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.371469 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.372103 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.372125 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.374570 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.378679 (+ 4109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.378684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.378689 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.379134 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.379630 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.379635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.379640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.379840 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.380329 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.380335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.380482 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.380606 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.380899 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:49:12.381445 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.367286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.29.174.253:40749)
0810 18:49:12.367433 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.367437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.367551 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:49:12.367681 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.367685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.367748 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.367822 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.368442 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.369226 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.370672 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.371152 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.371246 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.374791 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.374796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.374799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.374830 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.378559 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.379248 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.379254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.379258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.379515 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.379948 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.379953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.379955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.380218 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.380396 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.380713 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.380961 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:49:12.382805 9525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.252@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
I20260810 18:49:12.400471 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.383537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.0.0.1:39902)
0810 18:49:12.383683 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.383699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.383789 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.384045 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.384048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.384361 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.384496 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.384503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.385343 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.385347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.386256 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.386268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.387016 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.387021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.387363 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.391618 (+ 4255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.391636 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.394294 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.398000 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.398004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.398006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.398617 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.398986 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.398990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.398992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.399226 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.399668 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.399672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.399818 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.399957 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.400142 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:49:12.400693 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.383715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.29.174.252:40381)
0810 18:49:12.383845 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.383849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.383862 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:49:12.383898 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.383901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.383952 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.384021 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.384603 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.385244 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.386400 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.386844 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.386955 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.394546 (+ 7591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.394552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.394555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.394584 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.397876 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.398716 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.398719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.398721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.398893 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.399317 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.399321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.399323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.399563 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.399666 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.400041 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.400287 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":47}
I20260810 18:49:12.406080 9525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Bootstrap starting.
I20260810 18:49:12.410633 9525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:12.412132 9525 log.cc:824] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:12.415047 9525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: No bootstrap required, opened a new log
I20260810 18:49:12.425083 9525 raft_consensus.cc:348] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } }
I20260810 18:49:12.425479 9525 raft_consensus.cc:374] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:12.425716 9525 raft_consensus.cc:729] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: defc7503518a41b0a103c6be9fc8b4ec, State: Initialized, Role: FOLLOWER
I20260810 18:49:12.426383 9525 consensus_queue.cc:260] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } }
I20260810 18:49:12.427712 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } } }
I20260810 18:49:12.428107 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:12.428668 9525 sys_catalog.cc:564] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:49:12.441974 9571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:49:12.442147 9571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:49:12.446848 9289 master_runner.cc:428] Detected that this master defc7503518a41b0a103c6be9fc8b4ec is joining an existing cluster
I20260810 18:49:12.447026 9289 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.174.252:40381
I20260810 18:49:12.459322 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 8489
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.193@KRBTEST.COM
I20260810 18:49:12.487274 9579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.453891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.0.0.1:39912)
0810 18:49:12.461138 (+ 7247us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.461156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.461304 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.461846 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.461852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.467947 (+ 6095us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.468702 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.468718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.470724 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.470730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.473623 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.473638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.475517 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.475526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.475830 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.476743 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.476765 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.478481 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.483662 (+ 5181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.483666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.483670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.484222 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.484716 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.484720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.484724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.484915 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.485737 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.485743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.485949 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.486090 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.486441 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7013,"thread_start_us":1351,"threads_started":1}
I20260810 18:49:12.488314 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.452691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39912 (local address 127.29.174.252:40381)
0810 18:49:12.452839 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.452843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.461251 (+ 8408us) server_negotiation.cc:407] Connection header received
0810 18:49:12.461442 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.461447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.461526 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.461639 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.468849 (+ 7210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.470407 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.473763 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.474747 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.474872 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.478664 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.478671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.478676 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.478708 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.483502 (+ 4794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.484362 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.484367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.484372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.484585 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.485038 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.485043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.485046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.485522 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.485677 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.487614 (+ 1937us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.487868 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:49:12.494187 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.461342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.29.174.193:39495)
0810 18:49:12.469235 (+ 7893us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.469245 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.469272 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:49:12.469338 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.469347 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.469606 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.472236 (+ 2630us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.472719 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.474754 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.477092 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.478239 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.479083 (+ 844us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.485347 (+ 6264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.485361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.485378 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.485416 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.490512 (+ 5096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.491397 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.491405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.491409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.491570 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.492047 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.492052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.492055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.492510 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.492942 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.493336 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.493606 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7638,"thread_start_us":9556,"threads_started":1}
I20260810 18:49:12.495800 8677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:12.498924 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.453653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56246)
0810 18:49:12.455763 (+ 2110us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.455785 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.456026 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.456640 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.456645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.466812 (+ 10167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.467430 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.467444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.469300 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.469304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.472263 (+ 2959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.472280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.473824 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.473832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.474058 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.474788 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.474810 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.476592 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.482290 (+ 5698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.482296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.482299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.494436 (+ 12137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.495132 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.495137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.495141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.495409 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.496300 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.496305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.497841 (+ 1536us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.497992 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.498375 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1863,"thread_start_us":6971,"threads_started":1}
I20260810 18:49:12.500025 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.452872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.29.174.254:37617)
0810 18:49:12.453045 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.453051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.455878 (+ 2827us) server_negotiation.cc:407] Connection header received
0810 18:49:12.456206 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.456212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.456345 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.456421 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.467584 (+ 11163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.469069 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.472441 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.473333 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.473437 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.476838 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.476844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.476848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.476889 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.482125 (+ 5236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.494711 (+ 12586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.494718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.494721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.494969 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.495556 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.495561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.495565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.496075 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.496234 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.499242 (+ 3008us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.499515 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:49:12.504524 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.454243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:37694)
0810 18:49:12.466505 (+ 12262us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.466523 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.467052 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.467666 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.467671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.471881 (+ 4210us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.472738 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.472752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.476856 (+ 4104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.476861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.479178 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.479194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.482417 (+ 3223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.482425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.482631 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.483316 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.483340 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.484681 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.500762 (+ 16081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.500766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.500769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.501413 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.501925 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.501930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.501933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.502163 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.502981 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.502986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.503286 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.503423 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.503721 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12118,"thread_start_us":12381,"threads_started":1}
W20260810 18:49:12.505278 9460 tablet.cc:2369] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:12.506088 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.453164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.29.174.253:40749)
0810 18:49:12.453352 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.453358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.466608 (+ 13250us) server_negotiation.cc:407] Connection header received
0810 18:49:12.467229 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.467235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.467313 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.467382 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.475249 (+ 7867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.476592 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.480854 (+ 4262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.481822 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.481923 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.496603 (+ 14680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.496608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.496611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.496648 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.500632 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.501561 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.501566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.501570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.501788 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.502305 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.502310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.502313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.502774 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.502920 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.505100 (+ 2180us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.505510 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387749, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.254@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
I20260810 18:49:12.528437 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.507856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.29.174.252:40381)
0810 18:49:12.508011 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.508016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.508033 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:49:12.508156 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.508161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.508244 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.508331 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.510400 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.512128 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.514130 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.515099 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.515253 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.520134 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.520164 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.520168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.520208 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.525297 (+ 5089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.526137 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.526143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.526147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.526298 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.526741 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.526746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.526749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.527082 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.527212 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.527786 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.528014 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:49:12.528733 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.506831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.0.0.1:39926)
0810 18:49:12.507033 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.507052 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.507158 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.508474 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.508479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.509570 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.510095 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.510111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.512290 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.512296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.513984 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.514001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.515299 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.515307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.515505 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.516251 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.516275 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.519845 (+ 3570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.525441 (+ 5596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.525446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.525450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.526024 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.526441 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.526447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.526450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.526625 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.527222 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.527227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.527521 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.527681 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.527997 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:49:12.530525 8392 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.174.252:40381: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec"
member_type: NON_VOTER
last_known_addr {
host: "127.29.174.252"
port: 40381
}
attrs {
promote: true
}
}
dest_uuid: "1f5fcde8ba364546992fca787dbc4e05"
cas_config_opid_index: 6
I20260810 18:49:12.531481 8392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } }
I20260810 18:49:12.534291 9370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 18:49:12.534652 9488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:12.535043 9537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 18:49:12.537943 9537 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:12.539343 9488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 18:49:12.542168 9537 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.174.252:40381
I20260810 18:49:12.542660 9527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:12.542799 9289 master.cc:537] Master@127.29.174.252:40381 shutting down...
I20260810 18:49:12.545853 9370 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
W20260810 18:49:12.546482 8371 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 18:49:12.546705 9527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:12.546962 9527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:12.547483 9537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:12.547822 9537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
W20260810 18:49:12.557435 8371 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 -> Peer defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252:40381): Couldn't send request to peer defc7503518a41b0a103c6be9fc8b4ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:49:12.560238 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:12.560531 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:12.572737 9550 raft_consensus.cc:482] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:12.573148 9550 raft_consensus.cc:504] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } }
I20260810 18:49:12.579404 9550 leader_election.cc:288] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617), 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253:40749)
I20260810 18:49:12.580132 9370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "defc7503518a41b0a103c6be9fc8b4ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c26ffbaced4013a95c6a6a108cce52" is_pre_election: true
I20260810 18:49:12.580169 8411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "defc7503518a41b0a103c6be9fc8b4ec" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5fcde8ba364546992fca787dbc4e05" is_pre_election: true
I20260810 18:49:12.581189 9451 leader_election.cc:302] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: defc7503518a41b0a103c6be9fc8b4ec; no voters: 1f5fcde8ba364546992fca787dbc4e05, 54c26ffbaced4013a95c6a6a108cce52
I20260810 18:49:12.581522 9550 raft_consensus.cc:2726] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:49:12.583659 9289 raft_consensus.cc:2229] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:12.583984 9289 raft_consensus.cc:2258] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:12.584192 9289 tablet_replica.cc:330] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: stopping tablet replica
I20260810 18:49:12.591377 9289 master.cc:552] Master@127.29.174.252:40381 shutdown complete.
I20260810 18:49:12.591473 9289 master_runner.cc:305] Clearing existing system tablet
I20260810 18:49:12.592269 9289 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.592487 9289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.595716 9289 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:49:12.597282 9591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.597587 9289 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:12.597891 9289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:49:12.598006 9289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "defc7503518a41b0a103c6be9fc8b4ec"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-6zbq"
server_key: "d40b546dc7b8be2e6b3b9b500c30587f"
server_key_iv: "f763b0cb1ca9fa1f37b13e36229b5581"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.601680 9289 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:12.609892 9289 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:49:12.610132 9289 log.cc:1196] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 18:49:12.610893 9289 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Deleting consensus metadata
I20260810 18:49:12.611167 9289 master_runner.cc:315] Copying system tablet from 127.29.174.254:37617
I20260810 18:49:12.620468 9289 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: tablet copy: Beginning tablet copy session from remote peer at address 127.29.174.254:37617
I20260810 18:49:12.635205 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.621278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.29.174.254:37617)
0810 18:49:12.621680 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:12.621687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:12.622707 (+ 1020us) server_negotiation.cc:407] Connection header received
0810 18:49:12.622930 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:12.622935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:12.623022 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:12.623105 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:12.624323 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.625101 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.626875 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.627312 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.627424 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.629739 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:12.629744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:12.629747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:12.629781 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:12.632338 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.633013 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.633018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.633020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.633158 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:12.633644 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:12.633649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:12.633653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:12.633970 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:12.634101 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:12.634561 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 18:49:12.634771 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260810 18:49:12.635515 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:12.621318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:56256)
0810 18:49:12.622588 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:12.622608 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:12.622794 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:12.623162 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:12.623166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:12.623836 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:12.624163 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.624175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.625237 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.625243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.626731 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:12.626745 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:12.627462 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:12.627468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:12.627780 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:12.628483 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:12.628502 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:12.629559 (+ 1057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:12.632457 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.632463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.632467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.632897 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.633270 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:12.633275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:12.633279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:12.633498 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:12.634098 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:12.634103 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:12.634264 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 18:49:12.634451 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 18:49:12.634786 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":758,"threads_started":1}
I20260810 18:49:12.636291 8431 tablet_copy_service.cc:140] P 1f5fcde8ba364546992fca787dbc4e05: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer defc7503518a41b0a103c6be9fc8b4ec ({username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:56256)
I20260810 18:49:12.636505 8431 tablet_copy_service.cc:161] P 1f5fcde8ba364546992fca787dbc4e05: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer defc7503518a41b0a103c6be9fc8b4ec at {username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:56256: session id = defc7503518a41b0a103c6be9fc8b4ec-00000000000000000000000000000000
I20260810 18:49:12.639649 8431 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:49:12.641387 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:49:12.652673 9289 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: tablet copy: Starting download of 0 data blocks...
I20260810 18:49:12.653002 9289 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: tablet copy: Starting download of 1 WAL segments...
I20260810 18:49:12.655277 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 8489
I20260810 18:49:12.657779 9289 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:49:12.662146 8431 tablet_copy_service.cc:342] P 1f5fcde8ba364546992fca787dbc4e05: Request end of tablet copy session defc7503518a41b0a103c6be9fc8b4ec-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:56256
I20260810 18:49:12.662428 8431 tablet_copy_service.cc:433] P 1f5fcde8ba364546992fca787dbc4e05: ending tablet copy session defc7503518a41b0a103c6be9fc8b4ec-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer defc7503518a41b0a103c6be9fc8b4ec
I20260810 18:49:12.680330 9289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:12.683104 9604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.693523 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:39495
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:37617
--webserver_port=44137
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:12.700553 9607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:12.703051 9605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:12.703737 9289 server_base.cc:1034] running on GCE node
I20260810 18:49:12.704098 9289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:12.704757 9289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:12.706024 9289 hybrid_clock.cc:648] HybridClock initialized: now 1786387752705986 us; error 32 us; skew 500 ppm
I20260810 18:49:12.706259 9289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:12 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387752, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:12.712260 9289 init.cc:377] Logged in from keytab as oryx/127.29.174.252@KRBTEST.COM (short username oryx)
I20260810 18:49:12.720841 9289 webserver.cc:460] Webserver started at http://127.29.174.252:35821/ using document root <none> and password file <none>
I20260810 18:49:12.721211 9289 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:12.721341 9289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:12.724467 9289 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:12.726468 9622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:12.726914 9289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:49:12.727103 9289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "defc7503518a41b0a103c6be9fc8b4ec"
format_stamp: "Formatted at 2026-08-10 18:49:12 on dist-test-slave-6zbq"
server_key: "d40b546dc7b8be2e6b3b9b500c30587f"
server_key_iv: "f763b0cb1ca9fa1f37b13e36229b5581"
server_key_version: "encryptionkey@0"
I20260810 18:49:12.727370 9289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:12.741955 9289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:12.746371 9289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:12.771860 9289 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.252:40381
I20260810 18:49:12.771885 9699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.252:40381 every 8 connection(s)
I20260810 18:49:12.773164 9289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
I20260810 18:49:12.778498 9701 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:49:12.782303 9701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Bootstrap starting.
I20260810 18:49:12.798940 9701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:49:12.799579 9701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Bootstrap complete.
I20260810 18:49:12.800693 9701 raft_consensus.cc:348] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } }
I20260810 18:49:12.800999 9701 raft_consensus.cc:729] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: defc7503518a41b0a103c6be9fc8b4ec, State: Initialized, Role: LEARNER
I20260810 18:49:12.801281 9701 consensus_queue.cc:260] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } }
I20260810 18:49:12.802172 9705 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:12.802541 9701 sys_catalog.cc:564] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:12.802794 9705 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: LEARNER
I20260810 18:49:12.821657 9723 catalog_manager.cc:1259] Loaded cluster ID: f9df7d5a26814eff8825735d74bc0e0f
I20260810 18:49:12.821794 9723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: loading cluster ID for follower catalog manager: success
I20260810 18:49:12.825927 9723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: acquiring CA information for follower catalog manager: success
I20260810 18:49:12.828635 9723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:49:13.128808 9609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:13.129206 9609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:13.129449 9609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:13.139884 9609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:13.140076 9609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:13.140344 9609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:13.153332 9609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:39495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=44137
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:13.154771 9609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:13.156244 9609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:49:13.168707 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.161055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.0.0.1:39936)
0810 18:49:13.161806 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.161826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.161968 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.162393 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.162398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.162430 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:13.162647 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.162664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.164139 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.164145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.165798 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.165814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.167718 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.167726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.167976 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.168003 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.168096 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.168244 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":484,"threads_started":1}
I20260810 18:49:13.168845 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.161292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.29.174.252:40381)
0810 18:49:13.162066 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.162071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.162088 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:49:13.162148 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.162151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.162220 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.162317 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:13.162788 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.164002 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.165983 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.167463 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.167673 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.167844 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.168143 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.168308 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":531,"threads_started":1}
I20260810 18:49:13.170455 9666 raft_consensus.cc:1203] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260810 18:49:13.171131 9705 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f5fcde8ba364546992fca787dbc4e05. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:13.171530 9705 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: LEARNER
W20260810 18:49:13.174330 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:13.174839 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:13.175208 9739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:13.176141 9609 server_base.cc:1034] running on GCE node
I20260810 18:49:13.176898 9609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:13.177868 9609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:13.179086 9609 hybrid_clock.cc:648] HybridClock initialized: now 1786387753179037 us; error 33 us; skew 500 ppm
I20260810 18:49:13.179446 9609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:13.183306 9609 init.cc:377] Logged in from keytab as oryx/127.29.174.193@KRBTEST.COM (short username oryx)
I20260810 18:49:13.185038 9609 webserver.cc:460] Webserver started at http://127.29.174.193:44137/ using document root <none> and password file <none>
I20260810 18:49:13.185818 9609 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:13.185961 9609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:13.191833 9609 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:49:13.195230 9748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.196750 9609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:49:13.196928 9609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
format_stamp: "Formatted at 2026-08-10 18:49:09 on dist-test-slave-6zbq"
server_key: "edef890e214b2fca8d221f60277ecbb8"
server_key_iv: "2b12b092fd51806db1c46116deae520f"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.198297 9609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:13.215070 9609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:13.220018 9609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:13.220373 9609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:13.221833 9609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:13.223526 9609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:13.223609 9609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.223693 9609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:13.223775 9609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.293210 9609 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:39495
I20260810 18:49:13.293308 9926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:39495 every 8 connection(s)
I20260810 18:49:13.294881 9609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:13.297866 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 9609
I20260810 18:49:13.298197 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 8691
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.194@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:13.344892 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.299794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.29.174.194:32895)
0810 18:49:13.316579 (+ 16785us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.316588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.316619 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:49:13.316694 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.316703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.316968 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.319499 (+ 2531us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.319883 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.321654 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.323429 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.324690 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.324833 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.333724 (+ 8891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.333741 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.333756 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.333826 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.338788 (+ 4962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.340923 (+ 2135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.340931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.340935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.341173 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.341750 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.341755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.341760 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.342334 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.342775 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.343965 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.344234 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16600,"thread_start_us":16400,"threads_started":1}
I20260810 18:49:13.346791 8889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:13.349016 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.299096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:37379 (local address 127.29.174.253:40749)
0810 18:49:13.312619 (+ 13523us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.312627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.317241 (+ 4614us) server_negotiation.cc:407] Connection header received
0810 18:49:13.323507 (+ 6266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.323512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.323642 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.323720 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.327408 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.328723 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.331566 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.332340 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.332471 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.339528 (+ 7057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.339535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.339540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.339577 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.344498 (+ 4921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.345496 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.345501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.345505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.345706 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.346699 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.346705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.346708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.347170 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.347354 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.348243 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.348472 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13363,"thread_start_us":13544,"threads_started":1}
I20260810 18:49:13.349164 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.300453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:43355 (local address 127.29.174.254:37617)
0810 18:49:13.315109 (+ 14656us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.315116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.319290 (+ 4174us) server_negotiation.cc:407] Connection header received
0810 18:49:13.323578 (+ 4288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.323584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.323659 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.323807 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.327609 (+ 3802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.329260 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.332405 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.333302 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.333425 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.339485 (+ 6060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.339492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.339496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.339544 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.344148 (+ 4604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.345373 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.345379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.345383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.345541 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.346653 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.346659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.346663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.347071 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.347206 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.348262 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.348551 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14479,"thread_start_us":14913,"threads_started":1}
I20260810 18:49:13.350308 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.315303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:59977 (local address 127.29.174.252:40381)
0810 18:49:13.315624 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.315630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.322930 (+ 7300us) server_negotiation.cc:407] Connection header received
0810 18:49:13.323505 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.323510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.323592 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.323678 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.327579 (+ 3901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.328877 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.331784 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.332718 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.332869 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.339437 (+ 6568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.339445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.339449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.339489 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.344634 (+ 5145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.346999 (+ 2365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.347005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.347008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.347232 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.347815 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.347820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.347824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.348253 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.348407 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.348963 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.349697 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260810 18:49:13.351773 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.299731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:43355)
0810 18:49:13.319121 (+ 19390us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.319149 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.323202 (+ 4053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.324214 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.324219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.326160 (+ 1941us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.327365 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.327386 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.329507 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.329512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.332266 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.332278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.333768 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.333775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.334049 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.334830 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.334856 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.339239 (+ 4383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.344305 (+ 5066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.344313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.344327 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.345254 (+ 927us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.345654 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.345660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.345663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.346482 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.347299 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.347306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.347770 (+ 464us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.349337 (+ 1567us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.349913 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14938,"mutex_wait_us":863,"spinlock_wait_cycles":21888,"thread_start_us":16540,"threads_started":1}
I20260810 18:49:13.351943 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.300826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.29.174.193:37379)
0810 18:49:13.317025 (+ 16199us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.317114 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.323196 (+ 6082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.324209 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.324217 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.326057 (+ 1840us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.327163 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.327199 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.329052 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.329057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.331434 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.331446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.332869 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.332878 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.333293 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.334295 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.334316 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.339245 (+ 4929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.344664 (+ 5419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.344669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.344672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.345352 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.345851 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.345856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.345859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.346555 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.347449 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.347454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.347794 (+ 340us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.349303 (+ 1509us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.350704 (+ 1401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15944,"mutex_wait_us":926,"spinlock_wait_cycles":77184,"thread_start_us":13402,"threads_started":1}
I20260810 18:49:13.353467 9930 heartbeater.cc:347] Connected to a master server at 127.29.174.253:40749
I20260810 18:49:13.353525 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.315684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.29.174.193:59977)
0810 18:49:13.322794 (+ 7110us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.322835 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.323203 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.324205 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.324217 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.326169 (+ 1952us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.327408 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.327419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.329054 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.329059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.331634 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.331650 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.333204 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.333212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.333449 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.334179 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.334248 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.339167 (+ 4919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.344772 (+ 5605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.344777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.344781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.346812 (+ 2031us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.347365 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.347369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.347373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.347671 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.348491 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.348496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.348656 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.350924 (+ 2268us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.351186 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6938,"mutex_wait_us":145,"thread_start_us":6666,"threads_started":1}
I20260810 18:49:13.353638 9929 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:13.353848 9930 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.354022 9929 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.354930 9928 heartbeater.cc:347] Connected to a master server at 127.29.174.252:40381
I20260810 18:49:13.355139 9928 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.355332 9930 heartbeater.cc:511] Master 127.29.174.253:40749 requested a full tablet report, sending...
I20260810 18:49:13.355361 9929 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:13.355873 9928 heartbeater.cc:511] Master 127.29.174.252:40381 requested a full tablet report, sending...
I20260810 18:49:13.357869 8392 ts_manager.cc:194] Re-registered known tserver with Master: 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:13.358224 9652 ts_manager.cc:194] Registered new tserver with Master: 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:13.359032 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:43355
I20260810 18:49:13.359939 9349 ts_manager.cc:194] Registered new tserver with Master: 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:13.444157 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 8691
I20260810 18:49:13.465198 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:32895
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:37617
--webserver_port=42201
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:49:13.585832 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: NON_VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: true } } }
I20260810 18:49:13.586222 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:13.588012 9949 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: attempting to promote NON_VOTER defc7503518a41b0a103c6be9fc8b4ec to VOTER
I20260810 18:49:13.588732 9949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } }
I20260810 18:49:13.591279 9666 raft_consensus.cc:1261] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 LEARNER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 18:49:13.591374 9370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Refusing update from remote peer 1f5fcde8ba364546992fca787dbc4e05: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 18:49:13.591869 9724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:49:13.592217 9949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:49:13.595986 9724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.596621 9370 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.599494 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.599735 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:13.599970 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.600163 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:13.600384 9724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.600584 9724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:13.600883 9958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.601325 9958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:13.601058 9666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.604627 9705 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5fcde8ba364546992fca787dbc4e05" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:13.604892 9705 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: FOLLOWER
W20260810 18:49:13.789348 9947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:13.789589 9947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:13.789696 9947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:13.796717 9947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:13.796825 9947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:13.796983 9947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:13.806588 9947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:32895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=42201
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:13.807766 9947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:13.808987 9947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:13.821521 9967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:13.821547 9963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:13.821453 9965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:13.822283 9947 server_base.cc:1034] running on GCE node
I20260810 18:49:13.822842 9947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:13.823635 9947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:13.824846 9947 hybrid_clock.cc:648] HybridClock initialized: now 1786387753824804 us; error 38 us; skew 500 ppm
I20260810 18:49:13.825115 9947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:13.828039 9947 init.cc:377] Logged in from keytab as oryx/127.29.174.194@KRBTEST.COM (short username oryx)
I20260810 18:49:13.829391 9947 webserver.cc:460] Webserver started at http://127.29.174.194:42201/ using document root <none> and password file <none>
I20260810 18:49:13.829999 9947 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:13.830124 9947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:13.835000 9947 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:13.837440 9979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.838660 9947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:13.838804 9947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "47c487ac956942a191e7b1aabc41d3da"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "417819400e78abf93114a9dc7897b1f3"
server_key_iv: "46edee3e5440d28ce7dd3712cdc334cb"
server_key_version: "encryptionkey@0"
I20260810 18:49:13.839741 9947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:13.869972 9947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:13.873834 9947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:13.874078 9947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:13.874998 9947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:13.876348 9947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:13.876441 9947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.876538 9947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:13.876608 9947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:13.906028 9947 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:32895
I20260810 18:49:13.906085 10161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:32895 every 8 connection(s)
I20260810 18:49:13.907322 9947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:13.916883 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 9947
I20260810 18:49:13.917173 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 8906
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387753, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
Aug 10 18:49:13 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.195@KRBTEST.COM
I20260810 18:49:13.942759 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.910690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:58111 (local address 127.29.174.252:40381)
0810 18:49:13.911248 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.911254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.918893 (+ 7639us) server_negotiation.cc:407] Connection header received
0810 18:49:13.918976 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.918982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.919065 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.919176 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.923179 (+ 4003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924647 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.927067 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.927916 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.927987 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.933950 (+ 5963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.933958 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.933963 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.934004 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.938394 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.939397 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.939402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.939406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.939615 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.940296 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.940300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.940304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.940758 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.940889 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.941984 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.942228 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":498,"threads_started":1}
I20260810 18:49:13.943971 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.911046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:39427 (local address 127.29.174.254:37617)
0810 18:49:13.911915 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.911921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.914092 (+ 2171us) server_negotiation.cc:407] Connection header received
0810 18:49:13.919532 (+ 5440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.919538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.919663 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.919792 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.922838 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924200 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.927737 (+ 3537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.928519 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.928644 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.934320 (+ 5676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.934328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.934333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.934401 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.939256 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.940042 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.940047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.940051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.940194 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.940688 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.940693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.940696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.941124 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.941303 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.943363 (+ 2060us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.943578 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":640,"threads_started":1}
I20260810 18:49:13.944267 10173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.912851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.29.174.194:58111)
0810 18:49:13.918125 (+ 5274us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.918146 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.918300 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.919457 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.919469 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.922514 (+ 3045us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.922996 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.923009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924947 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.924952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.926886 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.926900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.928309 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.928317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.928611 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.929536 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.929574 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.933629 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.938553 (+ 4924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.938562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.938575 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.939269 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.939836 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.939841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.939845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.940164 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.940985 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.940992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.941423 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.943053 (+ 1630us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.943299 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5101,"mutex_wait_us":1,"thread_start_us":4738,"threads_started":1}
I20260810 18:49:13.944432 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.911742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:39427)
0810 18:49:13.913935 (+ 2193us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.913991 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.917375 (+ 3384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.919838 (+ 2463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.919844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.921094 (+ 1250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.922596 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.922626 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924384 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.924389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.927612 (+ 3223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.927629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.928773 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.928781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.929367 (+ 586us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.930126 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.930148 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.934083 (+ 3935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.939396 (+ 5313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.939400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.939404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.939900 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.940328 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.940332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.940335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.940545 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.941368 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.941373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.941543 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.941816 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.942417 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1813,"mutex_wait_us":107,"thread_start_us":1404,"threads_started":1}
I20260810 18:49:13.945689 10163 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:13.946014 10163 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.946429 10162 heartbeater.cc:347] Connected to a master server at 127.29.174.252:40381
I20260810 18:49:13.946623 10162 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.947273 10163 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:13.947511 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.913005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.29.174.194:51355)
0810 18:49:13.918914 (+ 5909us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:13.918929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:13.919093 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:13.920492 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:13.920498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:13.921277 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:13.922597 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.922628 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924470 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.924475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.927330 (+ 2855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:13.927344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.930146 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.930154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:13.930469 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.931185 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:13.931208 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:13.937652 (+ 6444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:13.943083 (+ 5431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.943100 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.943104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.943940 (+ 836us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.944395 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:13.944401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:13.944405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:13.944924 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:13.946131 (+ 1207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:13.946136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:13.946300 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 18:49:13.946482 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 18:49:13.946748 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5807,"mutex_wait_us":118,"thread_start_us":6963,"threads_started":1}
I20260810 18:49:13.947793 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.911194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:51355 (local address 127.29.174.253:40749)
0810 18:49:13.914770 (+ 3576us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.914775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.919002 (+ 4227us) server_negotiation.cc:407] Connection header received
0810 18:49:13.920165 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.920171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.920270 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.921375 (+ 1105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.922825 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.924289 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.927560 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.929493 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.929604 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.937907 (+ 8303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.937916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.937921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.937963 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.942943 (+ 4980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.944102 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.944108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.944112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.944254 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.945062 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.945067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.945071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.945993 (+ 922us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.946195 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.947054 (+ 859us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.947280 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3451,"thread_start_us":3271,"threads_started":1}
I20260810 18:49:13.948509 10162 heartbeater.cc:511] Master 127.29.174.252:40381 requested a full tablet report, sending...
I20260810 18:49:13.949180 8392 ts_manager.cc:194] Re-registered known tserver with Master: 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:13.949298 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:13.918736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.29.174.195:34889)
0810 18:49:13.921041 (+ 2305us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:13.921048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:13.921091 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:49:13.921190 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:13.921199 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:13.921466 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:13.921658 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:13.925254 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.927268 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.929574 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:13.930567 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:13.930701 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:13.938850 (+ 8149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:13.938867 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:13.938882 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:13.938947 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:13.944295 (+ 5348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.945555 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.945562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.945565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.945754 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:13.946195 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:13.946200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:13.946203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:13.946673 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:13.947012 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:13.948240 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0810 18:49:13.948813 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2114,"thread_start_us":1766,"threads_started":1}
I20260810 18:49:13.949792 9652 ts_manager.cc:194] Registered new tserver with Master: 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:13.950421 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:39427
I20260810 18:49:13.950695 9109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:13.951164 10165 heartbeater.cc:347] Connected to a master server at 127.29.174.253:40749
I20260810 18:49:13.951331 10165 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:13.951933 10165 heartbeater.cc:511] Master 127.29.174.253:40749 requested a full tablet report, sending...
I20260810 18:49:13.953119 9349 ts_manager.cc:194] Registered new tserver with Master: 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:14.036135 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 8906
I20260810 18:49:14.058460 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:34889
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:37617
--webserver_port=43791
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:49:14.361617 9929 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
W20260810 18:49:14.386059 10180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:14.386368 10180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:14.386543 10180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:14.394654 10180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:14.394760 10180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:14.394932 10180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:14.406654 10180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:34889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=43791
--tserver_master_addrs=127.29.174.254:37617,127.29.174.253:40749,127.29.174.252:40381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:14.408097 10180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:14.409482 10180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:14.423353 10193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:14.423382 10198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:14.423980 10196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:14.425253 10180 server_base.cc:1034] running on GCE node
I20260810 18:49:14.425886 10180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:14.427042 10180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:14.428274 10180 hybrid_clock.cc:648] HybridClock initialized: now 1786387754428208 us; error 52 us; skew 500 ppm
I20260810 18:49:14.428634 10180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:14 dist-test-slave-6zbq krb5kdc[8321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387754, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:14.431918 10180 init.cc:377] Logged in from keytab as oryx/127.29.174.195@KRBTEST.COM (short username oryx)
I20260810 18:49:14.433751 10180 webserver.cc:460] Webserver started at http://127.29.174.195:43791/ using document root <none> and password file <none>
I20260810 18:49:14.434526 10180 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:14.434679 10180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:14.440521 10180 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:49:14.443533 10208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:14.445425 10180 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:49:14.445591 10180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "69e1e6de0bd9433aac8cde7a713a6337"
format_stamp: "Formatted at 2026-08-10 18:49:10 on dist-test-slave-6zbq"
server_key: "9ff8ab9ddd21e93efff7e4513c39d828"
server_key_iv: "65cecf16b7c221d7374a057963588dbd"
server_key_version: "encryptionkey@0"
I20260810 18:49:14.446713 10180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:14.477982 10180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:14.483808 10180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:14.484127 10180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:14.485204 10180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:14.486735 10180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:14.486842 10180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:14.486953 10180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:14.487028 10180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:14.525594 10180 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34889
I20260810 18:49:14.525662 10394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34889 every 8 connection(s)
I20260810 18:49:14.527341 10180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:14.532764 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 10180
Aug 10 18:49:14 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387754, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.252@KRBTEST.COM
Aug 10 18:49:14 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387754, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
Aug 10 18:49:14 dist-test-slave-6zbq krb5kdc[8321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387754, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.253@KRBTEST.COM
I20260810 18:49:14.586117 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.531677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:52229)
0810 18:49:14.553970 (+ 22293us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.553992 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.554140 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.556625 (+ 2485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.556629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.557275 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:14.557568 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.557581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.559242 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.559251 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.560872 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.560888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.562227 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.562235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.562578 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.563331 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:14.563351 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:14.569115 (+ 5764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:14.576229 (+ 7114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.576250 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.576288 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.578011 (+ 1723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.579069 (+ 1058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.579077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.579080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.579618 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.581643 (+ 2025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:14.581659 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:14.582607 (+ 948us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.583191 (+ 584us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.583998 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22104,"mutex_wait_us":73,"spinlock_wait_cycles":32000,"thread_start_us":21894,"threads_started":1}
I20260810 18:49:14.588202 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.530862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:52229 (local address 127.29.174.254:37617)
0810 18:49:14.549467 (+ 18605us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.549475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.556264 (+ 6789us) server_negotiation.cc:407] Connection header received
0810 18:49:14.556344 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.556349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.556441 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.557730 (+ 1289us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.557924 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.559034 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.561105 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.561919 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.562863 (+ 944us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.569400 (+ 6537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.569408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.569412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.569473 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.575794 (+ 6321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.578300 (+ 2506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.578311 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.578316 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.578836 (+ 520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.579808 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.579815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.579818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.580443 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.580947 (+ 504us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.587101 (+ 6154us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.587505 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18423,"thread_start_us":18619,"threads_started":1}
I20260810 18:49:14.590420 10398 heartbeater.cc:347] Connected to a master server at 127.29.174.254:37617
I20260810 18:49:14.590941 10398 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:14.592164 10398 heartbeater.cc:511] Master 127.29.174.254:37617 requested a full tablet report, sending...
I20260810 18:49:14.593134 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.530624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56465 (local address 127.29.174.253:40749)
0810 18:49:14.553191 (+ 22567us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.553197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.553227 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:49:14.553312 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.553317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.553433 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.553610 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.556293 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.557471 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.560467 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.561157 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.561266 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.570371 (+ 9105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.570379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.570383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.570435 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.576426 (+ 5991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.587300 (+ 10874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.587310 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.587313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.587574 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.589999 (+ 2425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.590005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.590009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.590919 (+ 910us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.591079 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.592228 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.592524 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22411,"thread_start_us":22269,"threads_started":1}
I20260810 18:49:14.594358 8392 ts_manager.cc:194] Re-registered known tserver with Master: 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889)
I20260810 18:49:14.594652 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.531686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.29.174.195:56465)
0810 18:49:14.540843 (+ 9157us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.540933 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.547471 (+ 6538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.554134 (+ 6663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.554152 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.555196 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:14.556053 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.556078 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.557713 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.557718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.560247 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.560265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.561657 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.561666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.561996 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.562963 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:14.563003 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:14.570077 (+ 7074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:14.576787 (+ 6710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.576794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.576799 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.586516 (+ 9717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.587786 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.587792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.587796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.588078 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.591224 (+ 3146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:14.591229 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:14.591443 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.593438 (+ 1995us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.593892 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8682,"mutex_wait_us":33,"thread_start_us":22568,"threads_started":1}
I20260810 18:49:14.595464 8392 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:52229
I20260810 18:49:14.596374 10401 heartbeater.cc:347] Connected to a master server at 127.29.174.253:40749
I20260810 18:49:14.596633 10401 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:14.597625 10401 heartbeater.cc:511] Master 127.29.174.253:40749 requested a full tablet report, sending...
I20260810 18:49:14.599558 9349 ts_manager.cc:194] Registered new tserver with Master: 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889)
I20260810 18:49:14.605057 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.531678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.29.174.195:56985)
0810 18:49:14.554745 (+ 23067us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.554767 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.554908 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.555444 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.555450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.556209 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:14.556575 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.556589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.558852 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.558858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.560796 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.560816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.562125 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.562133 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.562408 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.563053 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:14.563077 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:14.567935 (+ 4858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:14.601233 (+ 33298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.601240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.601245 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.601901 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.602504 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:14.602508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:14.602511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:14.602773 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:14.603619 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:14.603624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:14.603883 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.604074 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.604481 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22871,"mutex_wait_us":2667,"thread_start_us":5764,"threads_started":1}
I20260810 18:49:14.606143 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.530968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56985 (local address 127.29.174.252:40381)
0810 18:49:14.544889 (+ 13921us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.544897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.554898 (+ 10001us) server_negotiation.cc:407] Connection header received
0810 18:49:14.555020 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.555025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.555124 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.555254 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.557132 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.558491 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.561060 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.561862 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.562770 (+ 908us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.568323 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.568332 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.568337 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.568388 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.600979 (+ 32591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.602114 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.602121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.602124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.602364 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.602928 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.602932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.602936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.603470 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.605334 (+ 1864us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.605524 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.605758 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13737,"thread_start_us":13600,"threads_started":1}
I20260810 18:49:14.618670 10395 heartbeater.cc:347] Connected to a master server at 127.29.174.252:40381
I20260810 18:49:14.619033 10395 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:14.620074 10395 heartbeater.cc:511] Master 127.29.174.252:40381 requested a full tablet report, sending...
I20260810 18:49:14.623133 9652 ts_manager.cc:194] Registered new tserver with Master: 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889)
I20260810 18:49:14.631691 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.566045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.29.174.254:37617)
0810 18:49:14.568874 (+ 2829us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.568880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.569772 (+ 892us) server_negotiation.cc:407] Connection header received
0810 18:49:14.570140 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.570146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.570233 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.570361 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.584980 (+ 14619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.587394 (+ 2414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.592497 (+ 5103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.593438 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.593582 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.603640 (+ 10058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.603650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.603655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.603705 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.620644 (+ 16939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.625090 (+ 4446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.625108 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.625111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.625952 (+ 841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.627205 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.627217 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.627221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.627909 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.628402 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.628937 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.629279 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2676,"thread_start_us":653,"threads_started":1}
I20260810 18:49:14.638548 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.578265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.29.174.252:40381)
0810 18:49:14.602402 (+ 24137us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.602408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.602435 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:49:14.602493 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.602497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.602579 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.602701 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.609154 (+ 6453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.611018 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.613695 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.614731 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.616139 (+ 1408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.625686 (+ 9547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.625710 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.625716 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.625998 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.634031 (+ 8033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.635811 (+ 1780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.635831 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.635837 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.636271 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.636846 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.636850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.636852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.637171 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.637440 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.637813 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.638037 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23945,"thread_start_us":24596,"threads_started":1}
I20260810 18:49:14.643437 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.571167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.29.174.253:40749)
0810 18:49:14.589266 (+ 18099us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.589272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.593007 (+ 3735us) server_negotiation.cc:407] Connection header received
0810 18:49:14.593064 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.593069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.593160 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.593265 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.606489 (+ 13224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.607867 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.623215 (+ 15348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.624219 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.624361 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.631388 (+ 7027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.631408 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.631413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.631498 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:14.638058 (+ 6560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.639866 (+ 1808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.639872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.639876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.640197 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:14.640756 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:14.640761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:14.640766 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:14.641248 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:14.641564 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.642448 (+ 884us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.642730 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17938,"thread_start_us":17708,"threads_started":1}
I20260810 18:49:14.641340 8392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:14.652845 8392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:14.695086 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.676215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.29.174.194:32895)
0810 18:49:14.677122 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.677132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.681364 (+ 4232us) server_negotiation.cc:407] Connection header received
0810 18:49:14.681738 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.681750 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.682094 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.682310 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:14.683870 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.685800 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.691340 (+ 5540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.693296 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.693463 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.693685 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.694457 (+ 772us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.694664 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":477,"threads_started":1}
I20260810 18:49:14.696220 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.675966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:32895 (local address 127.0.0.1:41576)
0810 18:49:14.681241 (+ 5275us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.681259 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.681510 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.682828 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.682833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.682861 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:14.683623 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.683635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.686022 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.686028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.688861 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.688875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.693982 (+ 5107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.693989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.694209 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.694246 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.695436 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.695584 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5063,"thread_start_us":5002,"threads_started":1}
I20260810 18:49:14.701295 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.681756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.29.174.193:39495)
0810 18:49:14.691181 (+ 9425us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.691192 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.692437 (+ 1245us) server_negotiation.cc:407] Connection header received
0810 18:49:14.692624 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.692630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.692828 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.693011 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:14.693974 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.695166 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.698519 (+ 3353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.699918 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.700120 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.700334 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.700642 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.700871 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9187,"thread_start_us":9576,"threads_started":1}
I20260810 18:49:14.700713 10060 tablet_service.cc:1467] Processing CreateTablet for tablet dff83a235456495cb8346ca48f5cd040 (DEFAULT_TABLE table=test-table [id=b63f49ed63cd4b2a9d56e938467b233e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:14.701382 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.681403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39495 (local address 127.0.0.1:35788)
0810 18:49:14.692325 (+ 10922us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.692342 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.692478 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.693020 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.693024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.693047 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:14.693784 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.693795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.695336 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.695341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.698335 (+ 2994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.698368 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.700225 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.700232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.700464 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.700490 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.700578 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.700701 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10748,"mutex_wait_us":3,"thread_start_us":10509,"threads_started":1}
I20260810 18:49:14.702142 10060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff83a235456495cb8346ca48f5cd040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:14.702747 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.675433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34889 (local address 127.0.0.1:36254)
0810 18:49:14.680002 (+ 4569us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.680026 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.680214 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.690788 (+ 10574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.690794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.690823 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:14.692078 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.692095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.695867 (+ 3772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.695872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.698310 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.698329 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.701672 (+ 3343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.701680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.701902 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.701927 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.702003 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.702090 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4368,"thread_start_us":4169,"threads_started":1}
I20260810 18:49:14.702859 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.675811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.29.174.195:34889)
0810 18:49:14.690053 (+ 14242us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.690063 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.690119 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:49:14.690228 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.690237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.690581 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.690800 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:14.692444 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.695702 (+ 3258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.698483 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.700601 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.700828 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.700994 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.702075 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.702327 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13968,"thread_start_us":13653,"threads_started":1}
I20260810 18:49:14.706197 10287 tablet_service.cc:1467] Processing CreateTablet for tablet dff83a235456495cb8346ca48f5cd040 (DEFAULT_TABLE table=test-table [id=b63f49ed63cd4b2a9d56e938467b233e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:14.706276 9830 tablet_service.cc:1467] Processing CreateTablet for tablet dff83a235456495cb8346ca48f5cd040 (DEFAULT_TABLE table=test-table [id=b63f49ed63cd4b2a9d56e938467b233e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:14.707537 10287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff83a235456495cb8346ca48f5cd040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:14.707791 9830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff83a235456495cb8346ca48f5cd040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:14.722656 10462 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Bootstrap starting.
I20260810 18:49:14.727087 10464 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Bootstrap starting.
I20260810 18:49:14.727465 10462 tablet_bootstrap.cc:654] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:14.729475 10463 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Bootstrap starting.
I20260810 18:49:14.729871 10462 log.cc:824] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:14.731971 10464 tablet_bootstrap.cc:654] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:14.734082 10464 log.cc:824] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:14.734149 10463 tablet_bootstrap.cc:654] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:14.736181 10463 log.cc:824] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:14.738330 10464 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: No bootstrap required, opened a new log
I20260810 18:49:14.738705 10464 ts_tablet_manager.cc:1397] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260810 18:49:14.739668 10463 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: No bootstrap required, opened a new log
I20260810 18:49:14.740021 10463 ts_tablet_manager.cc:1397] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260810 18:49:14.740166 10462 tablet_bootstrap.cc:492] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: No bootstrap required, opened a new log
I20260810 18:49:14.740517 10462 ts_tablet_manager.cc:1397] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20260810 18:49:14.748651 10464 raft_consensus.cc:348] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.749013 10464 raft_consensus.cc:374] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:14.749159 10464 raft_consensus.cc:729] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3cf7bd132414eac2906a80eeb9ed7, State: Initialized, Role: FOLLOWER
I20260810 18:49:14.749799 10464 consensus_queue.cc:260] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.752215 10464 ts_tablet_manager.cc:1428] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260810 18:49:14.753947 10463 raft_consensus.cc:348] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.754587 10463 raft_consensus.cc:374] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:14.754750 10463 raft_consensus.cc:729] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47c487ac956942a191e7b1aabc41d3da, State: Initialized, Role: FOLLOWER
I20260810 18:49:14.754777 10462 raft_consensus.cc:348] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.755281 10462 raft_consensus.cc:374] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:14.755411 10462 raft_consensus.cc:729] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e1e6de0bd9433aac8cde7a713a6337, State: Initialized, Role: FOLLOWER
I20260810 18:49:14.755457 10463 consensus_queue.cc:260] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.756304 10462 consensus_queue.cc:260] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.763540 10463 ts_tablet_manager.cc:1428] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Time spent starting tablet: real 0.023s user 0.017s sys 0.007s
I20260810 18:49:14.763535 10462 ts_tablet_manager.cc:1428] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Time spent starting tablet: real 0.023s user 0.012s sys 0.011s
I20260810 18:49:14.763840 10398 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
I20260810 18:49:14.763974 10163 heartbeater.cc:503] Master 127.29.174.254:37617 was elected leader, sending a full tablet report...
W20260810 18:49:14.780786 10402 tablet.cc:2369] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:49:14.822207 9931 tablet.cc:2369] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:14.829804 10470 raft_consensus.cc:482] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:14.830145 10470 raft_consensus.cc:504] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.832573 10470 leader_election.cc:288] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889), 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:14.843104 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.833143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:50253 (local address 127.29.174.195:34889)
0810 18:49:14.833627 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.833634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.834320 (+ 686us) server_negotiation.cc:407] Connection header received
0810 18:49:14.834598 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.834601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.834662 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.834749 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:14.835776 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.837011 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.840466 (+ 3455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.842035 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.842219 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.842300 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.842696 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.842815 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260810 18:49:14.843299 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.833140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:57957 (local address 127.29.174.193:39495)
0810 18:49:14.833391 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.833397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.834756 (+ 1359us) server_negotiation.cc:407] Connection header received
0810 18:49:14.834929 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.834934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.834994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.835060 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:14.836018 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.837295 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.840540 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.842106 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.842267 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.842332 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.842826 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.842972 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:49:14.843536 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.832840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39495 (local address 127.29.174.194:57957)
0810 18:49:14.834663 (+ 1823us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.834677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.834783 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.835116 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.835120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.835143 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:14.835799 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.835812 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.837445 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.837451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.840380 (+ 2929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.840391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.842406 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.842414 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.842679 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.842710 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.842786 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.842934 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1715,"mutex_wait_us":72,"thread_start_us":1565,"threads_started":1}
I20260810 18:49:14.843801 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.832825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34889 (local address 127.29.174.194:50253)
0810 18:49:14.834149 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:14.834231 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:14.834448 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:14.834780 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:14.834785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:14.834807 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:14.835620 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.835635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.837170 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.837174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.840313 (+ 3139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:14.840331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.842277 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.842288 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:14.842523 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.842562 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:49:14.842677 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:49:14.842803 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":26,"thread_start_us":692,"threads_started":1}
I20260810 18:49:14.844740 9861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff83a235456495cb8346ca48f5cd040" candidate_uuid: "47c487ac956942a191e7b1aabc41d3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" is_pre_election: true
I20260810 18:49:14.844836 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff83a235456495cb8346ca48f5cd040" candidate_uuid: "47c487ac956942a191e7b1aabc41d3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e1e6de0bd9433aac8cde7a713a6337" is_pre_election: true
I20260810 18:49:14.845369 9861 raft_consensus.cc:2454] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47c487ac956942a191e7b1aabc41d3da in term 0.
I20260810 18:49:14.845535 10326 raft_consensus.cc:2454] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47c487ac956942a191e7b1aabc41d3da in term 0.
I20260810 18:49:14.846268 9983 leader_election.cc:302] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47c487ac956942a191e7b1aabc41d3da, 69e1e6de0bd9433aac8cde7a713a6337; no voters:
I20260810 18:49:14.846771 10470 raft_consensus.cc:2781] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:14.846932 10470 raft_consensus.cc:482] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:14.847040 10470 raft_consensus.cc:3037] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:14.850049 10470 raft_consensus.cc:504] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.850965 10470 leader_election.cc:288] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [CANDIDATE]: Term 1 election: Requested vote from peers 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889), 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495)
I20260810 18:49:14.851544 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff83a235456495cb8346ca48f5cd040" candidate_uuid: "47c487ac956942a191e7b1aabc41d3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e1e6de0bd9433aac8cde7a713a6337"
I20260810 18:49:14.851544 9861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff83a235456495cb8346ca48f5cd040" candidate_uuid: "47c487ac956942a191e7b1aabc41d3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
I20260810 18:49:14.851784 10326 raft_consensus.cc:3037] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:14.851805 9861 raft_consensus.cc:3037] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:14.855662 9861 raft_consensus.cc:2454] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47c487ac956942a191e7b1aabc41d3da in term 1.
I20260810 18:49:14.856086 10326 raft_consensus.cc:2454] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47c487ac956942a191e7b1aabc41d3da in term 1.
I20260810 18:49:14.856201 9980 leader_election.cc:302] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47c487ac956942a191e7b1aabc41d3da, 52f3cf7bd132414eac2906a80eeb9ed7; no voters:
I20260810 18:49:14.856750 10470 raft_consensus.cc:2781] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:14.857620 10470 raft_consensus.cc:686] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 1 LEADER]: Becoming Leader. State: Replica: 47c487ac956942a191e7b1aabc41d3da, State: Running, Role: LEADER
I20260810 18:49:14.858425 10470 consensus_queue.cc:237] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } }
I20260810 18:49:14.864567 8392 catalog_manager.cc:5495] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da reported cstate change: term changed from 0 to 1, leader changed from <none> to 47c487ac956942a191e7b1aabc41d3da (127.29.174.194). New cstate: current_term: 1 leader_uuid: "47c487ac956942a191e7b1aabc41d3da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:14.885164 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.879060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.29.174.194:32895)
0810 18:49:14.879207 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.879211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.879226 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:49:14.879375 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.879378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.879483 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.879599 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:14.880019 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.881016 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.882690 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.883323 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.883439 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.884006 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:14.884503 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:14.884642 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:14.884742 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 18:49:14.884823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:49:14.894439 9861 raft_consensus.cc:1261] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Refusing update from remote peer 47c487ac956942a191e7b1aabc41d3da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:14.894675 10326 raft_consensus.cc:1261] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Refusing update from remote peer 47c487ac956942a191e7b1aabc41d3da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:14.895541 10478 consensus_queue.cc:1035] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:14.896083 10470 consensus_queue.cc:1035] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:14.920334 10483 mvcc.cc:205] Tried to move back new op lower bound from 7317044244037898240 to 7317044243904544768. Current Snapshot: MvccSnapshot[applied={T|T < 7317044244037898240}]
I20260810 18:49:14.945652 8392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:14.946000 8392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:14.952428 8392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f5fcde8ba364546992fca787dbc4e05: Sending DeleteTablet for 3 replicas of tablet dff83a235456495cb8346ca48f5cd040
I20260810 18:49:14.954133 9830 tablet_service.cc:1514] Processing DeleteTablet for tablet dff83a235456495cb8346ca48f5cd040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:14 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:35788
I20260810 18:49:14.954485 10287 tablet_service.cc:1514] Processing DeleteTablet for tablet dff83a235456495cb8346ca48f5cd040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:14 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36254
I20260810 18:49:14.955564 10060 tablet_service.cc:1514] Processing DeleteTablet for tablet dff83a235456495cb8346ca48f5cd040 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:14 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:41576
I20260810 18:49:14.967253 10498 tablet_replica.cc:330] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: stopping tablet replica
I20260810 18:49:14.967749 10498 raft_consensus.cc:2229] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:14.968392 10498 raft_consensus.cc:2258] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:14.968276 10501 tablet_replica.cc:330] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: stopping tablet replica
I20260810 18:49:14.970783 10498 ts_tablet_manager.cc:1905] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:14.970964 10501 raft_consensus.cc:2229] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:14.971885 10501 raft_consensus.cc:2258] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:14.974491 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 8345
I20260810 18:49:14.975229 10501 ts_tablet_manager.cc:1905] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:14.978381 8444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:14.981468 10498 ts_tablet_manager.cc:1918] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:14.981793 10498 log.cc:1196] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/dff83a235456495cb8346ca48f5cd040
I20260810 18:49:14.982707 10498 ts_tablet_manager.cc:1939] T dff83a235456495cb8346ca48f5cd040 P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting consensus metadata
I20260810 18:49:14.988761 10499 tablet_replica.cc:330] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: stopping tablet replica
I20260810 18:49:14.988901 10501 ts_tablet_manager.cc:1918] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:14.989076 10501 log.cc:1196] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/dff83a235456495cb8346ca48f5cd040
I20260810 18:49:14.989481 10499 raft_consensus.cc:2229] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:14.989828 10501 ts_tablet_manager.cc:1939] T dff83a235456495cb8346ca48f5cd040 P 47c487ac956942a191e7b1aabc41d3da: Deleting consensus metadata
I20260810 18:49:14.990221 10499 raft_consensus.cc:2258] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:14.992499 10499 ts_tablet_manager.cc:1905] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:15.001412 10499 ts_tablet_manager.cc:1918] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:15.001611 10499 log.cc:1196] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/dff83a235456495cb8346ca48f5cd040
I20260810 18:49:15.002305 10499 ts_tablet_manager.cc:1939] T dff83a235456495cb8346ca48f5cd040 P 69e1e6de0bd9433aac8cde7a713a6337: Deleting consensus metadata
I20260810 18:49:15.007017 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.957363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.29.174.253:40749)
0810 18:49:14.957575 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.957582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:14.972275 (+ 14693us) server_negotiation.cc:407] Connection header received
0810 18:49:14.978256 (+ 5981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:14.978262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:14.978380 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:14.978462 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:14.986526 (+ 8064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.987824 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.990533 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:14.991634 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:14.991737 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:14.996983 (+ 5246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:14.996992 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:14.996997 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:14.997042 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:15.002754 (+ 5712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.003787 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.003793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.003797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.003996 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.004959 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.004964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.004968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.005340 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:15.005655 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:15.006386 (+ 731us) server_negotiation.cc:299] Negotiation successful
0810 18:49:15.006639 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:49:15.026463 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:14.973223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.29.174.252:40381)
0810 18:49:14.973424 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:14.973430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:15.007179 (+ 33749us) server_negotiation.cc:407] Connection header received
0810 18:49:15.007465 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:15.007472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:15.007556 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:15.007627 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:15.010290 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.011655 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.013859 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.014819 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.014963 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:15.018973 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:15.018981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:15.018986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:15.019030 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:15.023150 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.024050 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.024056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.024059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.024199 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.024703 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.024708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.024711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.025083 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:15.025398 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:15.025738 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:49:15.025990 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:49:15.126852 8368 catalog_manager.cc:4841] TS 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889): tablet dff83a235456495cb8346ca48f5cd040 (table test-table [id=b63f49ed63cd4b2a9d56e938467b233e]) successfully deleted
I20260810 18:49:15.127849 8365 catalog_manager.cc:4841] TS 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495): tablet dff83a235456495cb8346ca48f5cd040 (table test-table [id=b63f49ed63cd4b2a9d56e938467b233e]) successfully deleted
W20260810 18:49:15.128070 8365 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.29.174.193:39495, user_credentials={real_user=slave}} blocked reactor thread for 143415us
I20260810 18:49:15.128579 8365 catalog_manager.cc:4841] TS 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895): tablet dff83a235456495cb8346ca48f5cd040 (table test-table [id=b63f49ed63cd4b2a9d56e938467b233e]) successfully deleted
I20260810 18:49:15.129184 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 8345
I20260810 18:49:15.181135 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:15.157572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.29.174.252:40381)
0810 18:49:15.157777 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:15.157783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:15.166609 (+ 8826us) server_negotiation.cc:407] Connection header received
0810 18:49:15.166864 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:15.166870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:15.166947 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:15.167014 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:15.169339 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.170476 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.171878 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.172449 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.172532 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:15.175628 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:15.175633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:15.175636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:15.175678 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:15.178291 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.178891 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.178896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.178898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.179013 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.179519 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.179524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.179527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.179875 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:15.180038 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:15.180595 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 18:49:15.180798 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:49:15.182583 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:15.157517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.29.174.253:40749)
0810 18:49:15.157937 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:15.157943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:15.165056 (+ 7113us) server_negotiation.cc:407] Connection header received
0810 18:49:15.165339 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:15.165345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:15.165434 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:15.165544 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:15.167903 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.168860 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.171009 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:15.171701 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:15.171820 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:15.175614 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:15.175621 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:15.175626 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:15.175669 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:15.179047 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.179796 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.179802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.179806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.179955 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:15.180347 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:15.180353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:15.180356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:15.180704 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:15.180936 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:15.181842 (+ 906us) server_negotiation.cc:299] Negotiation successful
0810 18:49:15.182050 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260810 18:49:15.928869 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:15.927756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:44921)
0810 18:49:15.928527 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:15.928654 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":434,"threads_started":1}
W20260810 18:49:15.929414 9929 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:37617 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:15.932324 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:15.931487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:56247)
0810 18:49:15.931965 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:15.932139 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":259,"threads_started":1}
W20260810 18:49:15.932840 10398 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:37617 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:15.935336 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:15.934615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:44149)
0810 18:49:15.935042 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:15.935179 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":221,"threads_started":1}
W20260810 18:49:15.935654 10163 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:37617 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.453154 10521 raft_consensus.cc:482] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f5fcde8ba364546992fca787dbc4e05)
I20260810 18:49:16.453356 10521 raft_consensus.cc:504] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } }
I20260810 18:49:16.454128 10521 leader_election.cc:288] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617), 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253:40749)
I20260810 18:49:16.460410 10522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f5fcde8ba364546992fca787dbc4e05)
I20260810 18:49:16.460700 10522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } }
I20260810 18:49:16.461272 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.454400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57152)
0810 18:49:16.460723 (+ 6323us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.461012 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6109,"threads_started":1}
W20260810 18:49:16.462476 9629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.464308 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.463638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57162)
0810 18:49:16.463902 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.464070 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260810 18:49:16.464567 9629 leader_election.cc:334] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617): Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.464942 10522 leader_election.cc:288] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617), defc7503518a41b0a103c6be9fc8b4ec (127.29.174.252:40381)
I20260810 18:49:16.466038 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.463806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57170)
0810 18:49:16.465527 (+ 1721us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.465737 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1604,"thread_start_us":1479,"threads_started":1}
W20260810 18:49:16.466770 9318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.472074 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.454441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:40749 (local address 127.0.0.1:35754)
0810 18:49:16.461523 (+ 7082us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.461540 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.461678 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.462777 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.462781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.462807 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.463080 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.463092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.465853 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.465859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.468710 (+ 2851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.468759 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.471163 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.471173 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.471392 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.471440 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.471520 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.471696 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6998,"thread_start_us":6686,"threads_started":1}
I20260810 18:49:16.473129 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.454534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.29.174.253:40749)
0810 18:49:16.462458 (+ 7924us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.462464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.462485 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:49:16.462552 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.462557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.462629 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.462737 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.463260 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.465663 (+ 2403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.469234 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.470887 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.472376 (+ 1489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.472466 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.472562 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.472727 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7771,"thread_start_us":7294,"threads_started":1}
I20260810 18:49:16.474414 9370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "defc7503518a41b0a103c6be9fc8b4ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54c26ffbaced4013a95c6a6a108cce52" is_pre_election: true
I20260810 18:49:16.474821 9370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate defc7503518a41b0a103c6be9fc8b4ec in term 1.
I20260810 18:49:16.474996 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.474230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57176)
0810 18:49:16.474427 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.474660 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 18:49:16.475144 9318 leader_election.cc:334] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617): Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.475364 9630 leader_election.cc:302] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54c26ffbaced4013a95c6a6a108cce52, defc7503518a41b0a103c6be9fc8b4ec; no voters: 1f5fcde8ba364546992fca787dbc4e05
I20260810 18:49:16.475716 10521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:49:16.475845 10521 raft_consensus.cc:482] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f5fcde8ba364546992fca787dbc4e05)
I20260810 18:49:16.475946 10521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:49:16.478466 10521 raft_consensus.cc:504] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } }
I20260810 18:49:16.479216 10521 leader_election.cc:288] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 election: Requested vote from peers 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617), 54c26ffbaced4013a95c6a6a108cce52 (127.29.174.253:40749)
I20260810 18:49:16.479749 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.479368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57190)
0810 18:49:16.479511 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.479617 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 18:49:16.479768 9370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "defc7503518a41b0a103c6be9fc8b4ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54c26ffbaced4013a95c6a6a108cce52"
I20260810 18:49:16.480015 9370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:49:16.480438 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.464431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52486 (local address 127.29.174.252:40381)
0810 18:49:16.474258 (+ 9827us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.474264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.474287 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:49:16.474374 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.474378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.474463 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.474561 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.475195 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.476480 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.478090 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.479397 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.479559 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.479629 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.480086 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.480218 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9705,"thread_start_us":9574,"threads_started":1}
I20260810 18:49:16.480463 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.463878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:40381 (local address 127.0.0.1:52486)
0810 18:49:16.466318 (+ 2440us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.466362 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.466479 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.474602 (+ 8123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.474611 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.474639 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.474956 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.474971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.476595 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.476599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.477925 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.477939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.479715 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.479721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.479915 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.479948 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.480056 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.480198 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2352,"thread_start_us":9824,"threads_started":1}
I20260810 18:49:16.481099 9666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c26ffbaced4013a95c6a6a108cce52" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "defc7503518a41b0a103c6be9fc8b4ec" is_pre_election: true
I20260810 18:49:16.481416 9666 raft_consensus.cc:2379] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54c26ffbaced4013a95c6a6a108cce52 in current term 2: Already voted for candidate defc7503518a41b0a103c6be9fc8b4ec in this term.
I20260810 18:49:16.482067 9324 leader_election.cc:302] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c26ffbaced4013a95c6a6a108cce52; no voters: 1f5fcde8ba364546992fca787dbc4e05, defc7503518a41b0a103c6be9fc8b4ec
I20260810 18:49:16.482658 9370 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate defc7503518a41b0a103c6be9fc8b4ec in term 2.
I20260810 18:49:16.482816 10522 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:49:16.483002 9630 leader_election.cc:302] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54c26ffbaced4013a95c6a6a108cce52, defc7503518a41b0a103c6be9fc8b4ec; no voters:
I20260810 18:49:16.483292 10521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:49:16.483781 10521 raft_consensus.cc:686] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [term 2 LEADER]: Becoming Leader. State: Replica: defc7503518a41b0a103c6be9fc8b4ec, State: Running, Role: LEADER
I20260810 18:49:16.483834 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.483442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57198)
0810 18:49:16.483567 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.483653 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:49:16.484014 9629 leader_election.cc:334] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617): Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
I20260810 18:49:16.484175 10521 consensus_queue.cc:237] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } }
I20260810 18:49:16.485922 10533 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader defc7503518a41b0a103c6be9fc8b4ec. Latest consensus state: current_term: 2 leader_uuid: "defc7503518a41b0a103c6be9fc8b4ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:16.486231 10533 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:16.486891 10536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:16.490761 10536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:16.491537 10536 catalog_manager.cc:1259] Loaded cluster ID: f9df7d5a26814eff8825735d74bc0e0f
I20260810 18:49:16.491618 10536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:16.492694 10536 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:16.493441 10536 catalog_manager.cc:5879] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Loaded TSK: 0
I20260810 18:49:16.494249 10536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:16.506309 9650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:16.513072 9650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:16.516542 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.516203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57226)
0810 18:49:16.516358 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.516422 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:49:16.516705 9370 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [term 2 FOLLOWER]: Refusing update from remote peer defc7503518a41b0a103c6be9fc8b4ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 18:49:16.517359 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.517065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57232)
0810 18:49:16.517191 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.517256 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:49:16.517395 10533 consensus_queue.cc:1035] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 18:49:16.517654 9629 consensus_peers.cc:487] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec -> Peer 1f5fcde8ba364546992fca787dbc4e05 (127.29.174.254:37617): Couldn't send request to peer 1f5fcde8ba364546992fca787dbc4e05. Status: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:49:16.519677 10522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader defc7503518a41b0a103c6be9fc8b4ec. Latest consensus state: current_term: 2 leader_uuid: "defc7503518a41b0a103c6be9fc8b4ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:16.519913 10522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:16.521296 10533 sys_catalog.cc:455] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "defc7503518a41b0a103c6be9fc8b4ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:16.521553 10533 sys_catalog.cc:458] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:16.521826 10539 mvcc.cc:205] Tried to move back new op lower bound from 7317044250686763008 to 7317044250562699264. Current Snapshot: MvccSnapshot[applied={T|T < 7317044250686763008}]
I20260810 18:49:16.523283 10522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "defc7503518a41b0a103c6be9fc8b4ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1f5fcde8ba364546992fca787dbc4e05" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 37617 } } peers { permanent_uuid: "54c26ffbaced4013a95c6a6a108cce52" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 40749 } attrs { promote: false } } peers { permanent_uuid: "defc7503518a41b0a103c6be9fc8b4ec" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 40381 } attrs { promote: false } } }
I20260810 18:49:16.523528 10522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c26ffbaced4013a95c6a6a108cce52 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:49:16.538859 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.531130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39495 (local address 127.0.0.1:57728)
0810 18:49:16.531264 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.531281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.531414 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.532417 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.532423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.532450 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.532680 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.532690 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.534334 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.534360 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.536396 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.536412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.538061 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.538068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.538270 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.538310 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.538395 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.538533 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 18:49:16.540138 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.531246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.29.174.193:39495)
0810 18:49:16.532071 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.532076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.532095 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:49:16.532169 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.532175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.532256 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.532822 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.532992 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.534163 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.536638 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.537876 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.539511 (+ 1635us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.539599 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.539664 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.539806 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":814,"threads_started":1}
I20260810 18:49:16.540863 9830 tablet_service.cc:1467] Processing CreateTablet for tablet 4e507ad65d8e40dfa2918d526f74489f (DEFAULT_TABLE table=test-table [id=41b8f471a6b04614a053722bb659a753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:16.541348 9830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e507ad65d8e40dfa2918d526f74489f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:16.547569 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.531883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34889 (local address 127.0.0.1:35512)
0810 18:49:16.539087 (+ 7204us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.539105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.539199 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.541459 (+ 2260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.541463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.541501 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.541688 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.541699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.543230 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.543235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.545215 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.545239 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.546761 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.546767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.546993 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.547027 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.547118 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.547248 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7100,"mutex_wait_us":69,"thread_start_us":14848,"threads_started":1}
I20260810 18:49:16.547642 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.532306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.29.174.195:34889)
0810 18:49:16.541138 (+ 8832us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.541143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.541162 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:49:16.541224 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.541230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.541306 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.541827 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.541997 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.543099 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.545384 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.546579 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.546743 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.546812 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.547166 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.547322 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8706,"thread_start_us":8613,"threads_started":1}
I20260810 18:49:16.548683 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.531476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:32895 (local address 127.0.0.1:33644)
0810 18:49:16.531602 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.531615 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.531688 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.541685 (+ 9997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.541689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.541715 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.541996 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.542004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.543071 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.543077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.545232 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.545242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.547997 (+ 2755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.548004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.548168 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.548200 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.548261 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.548383 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":69}
I20260810 18:49:16.548836 10287 tablet_service.cc:1467] Processing CreateTablet for tablet 4e507ad65d8e40dfa2918d526f74489f (DEFAULT_TABLE table=test-table [id=41b8f471a6b04614a053722bb659a753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:16.548947 10556 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Bootstrap starting.
I20260810 18:49:16.549192 10287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e507ad65d8e40dfa2918d526f74489f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:16.550097 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.531828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.29.174.194:32895)
0810 18:49:16.541398 (+ 9570us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.541404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.541423 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:49:16.541492 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.541497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.541561 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.541650 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.542129 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.542948 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.546449 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.547510 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.549495 (+ 1985us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.549565 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.549614 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.549762 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9474,"thread_start_us":9355,"threads_started":1}
I20260810 18:49:16.550858 10556 tablet_bootstrap.cc:654] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:16.551133 10060 tablet_service.cc:1467] Processing CreateTablet for tablet 4e507ad65d8e40dfa2918d526f74489f (DEFAULT_TABLE table=test-table [id=41b8f471a6b04614a053722bb659a753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:16.551568 10060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e507ad65d8e40dfa2918d526f74489f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:16.552708 10556 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: No bootstrap required, opened a new log
I20260810 18:49:16.552835 10556 ts_tablet_manager.cc:1397] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 18:49:16.553493 10556 raft_consensus.cc:348] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.553706 10556 raft_consensus.cc:374] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:16.554003 10556 raft_consensus.cc:729] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f3cf7bd132414eac2906a80eeb9ed7, State: Initialized, Role: FOLLOWER
I20260810 18:49:16.554586 10556 consensus_queue.cc:260] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.555574 10563 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Bootstrap starting.
I20260810 18:49:16.557277 10564 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Bootstrap starting.
I20260810 18:49:16.557336 10563 tablet_bootstrap.cc:654] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:16.559168 10564 tablet_bootstrap.cc:654] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:16.560976 10564 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: No bootstrap required, opened a new log
I20260810 18:49:16.561126 10564 ts_tablet_manager.cc:1397] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 18:49:16.561702 10564 raft_consensus.cc:348] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.561872 10564 raft_consensus.cc:374] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:16.561972 10564 raft_consensus.cc:729] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47c487ac956942a191e7b1aabc41d3da, State: Initialized, Role: FOLLOWER
I20260810 18:49:16.562184 10564 consensus_queue.cc:260] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.562402 10556 ts_tablet_manager.cc:1428] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Time spent starting tablet: real 0.009s user 0.000s sys 0.008s
I20260810 18:49:16.566854 9928 heartbeater.cc:503] Master 127.29.174.252:40381 was elected leader, sending a full tablet report...
I20260810 18:49:16.562960 10564 ts_tablet_manager.cc:1428] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:16.569276 10162 heartbeater.cc:503] Master 127.29.174.252:40381 was elected leader, sending a full tablet report...
I20260810 18:49:16.573462 10563 tablet_bootstrap.cc:492] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: No bootstrap required, opened a new log
I20260810 18:49:16.573588 10563 ts_tablet_manager.cc:1397] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.011s
I20260810 18:49:16.574262 10563 raft_consensus.cc:348] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.574496 10563 raft_consensus.cc:374] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:16.574604 10563 raft_consensus.cc:729] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e1e6de0bd9433aac8cde7a713a6337, State: Initialized, Role: FOLLOWER
I20260810 18:49:16.574865 10563 consensus_queue.cc:260] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.575418 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.563244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:33763)
0810 18:49:16.575005 (+ 11761us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.575165 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11639,"thread_start_us":11509,"threads_started":1}
I20260810 18:49:16.575639 10563 ts_tablet_manager.cc:1428] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:49:16.576426 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.566894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:52549)
0810 18:49:16.575783 (+ 8889us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.575946 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8759,"thread_start_us":7786,"threads_started":1}
I20260810 18:49:16.576815 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.576512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:59349)
0810 18:49:16.576646 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.576717 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 18:49:16.577409 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.577064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:59601)
0810 18:49:16.577203 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.577282 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 18:49:16.579080 10395 heartbeater.cc:503] Master 127.29.174.252:40381 was elected leader, sending a full tablet report...
I20260810 18:49:16.579238 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.575924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:37495)
0810 18:49:16.578846 (+ 2922us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.578981 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2780,"thread_start_us":2658,"threads_started":1}
I20260810 18:49:16.580308 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.579999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:51657)
0810 18:49:16.580150 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.580213 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:49:16.674641 10166 tablet.cc:2369] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:16.908866 10572 raft_consensus.cc:482] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:16.909091 10572 raft_consensus.cc:504] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.910956 10572 leader_election.cc:288] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495), 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:16.918764 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.911351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:33731 (local address 127.29.174.193:39495)
0810 18:49:16.911544 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.911549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.912169 (+ 620us) server_negotiation.cc:407] Connection header received
0810 18:49:16.912494 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.912501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.912582 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.912670 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.913303 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.914003 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.916518 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.917702 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.917868 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.917941 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.918397 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.918525 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:49:16.918782 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.911835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:33557 (local address 127.29.174.194:32895)
0810 18:49:16.912019 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:16.912024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:16.912169 (+ 145us) server_negotiation.cc:407] Connection header received
0810 18:49:16.912495 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:16.912499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:16.912557 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:16.912629 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:16.913137 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.914218 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.916518 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.917708 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.917868 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.917941 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:16.918396 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 18:49:16.918521 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:49:16.918918 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.911129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39495 (local address 127.29.174.195:33731)
0810 18:49:16.912060 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.912080 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.912310 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.912813 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.912817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.912836 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.913147 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.913158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.914135 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.914141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.916356 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.916365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.917912 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.917918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.918188 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.918256 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.918376 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.918531 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":107,"spinlock_wait_cycles":5632}
I20260810 18:49:16.919291 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.911469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:32895 (local address 127.29.174.195:33557)
0810 18:49:16.912081 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.912094 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:16.912310 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:16.912673 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:16.912678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:16.912704 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:16.912982 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.912998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.914371 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.914376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.916347 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:16.916364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:16.917912 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:16.917918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:16.918192 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:16.918256 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:49:16.918376 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:49:16.918539 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":103,"spinlock_wait_cycles":15744,"thread_start_us":562,"threads_started":1}
I20260810 18:49:16.919526 9861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e507ad65d8e40dfa2918d526f74489f" candidate_uuid: "69e1e6de0bd9433aac8cde7a713a6337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" is_pre_election: true
I20260810 18:49:16.919756 9861 raft_consensus.cc:2454] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69e1e6de0bd9433aac8cde7a713a6337 in term 0.
I20260810 18:49:16.919986 10092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e507ad65d8e40dfa2918d526f74489f" candidate_uuid: "69e1e6de0bd9433aac8cde7a713a6337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47c487ac956942a191e7b1aabc41d3da" is_pre_election: true
I20260810 18:49:16.920240 10211 leader_election.cc:302] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52f3cf7bd132414eac2906a80eeb9ed7, 69e1e6de0bd9433aac8cde7a713a6337; no voters:
I20260810 18:49:16.920324 10092 raft_consensus.cc:2454] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69e1e6de0bd9433aac8cde7a713a6337 in term 0.
I20260810 18:49:16.920595 10572 raft_consensus.cc:2781] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:16.920724 10572 raft_consensus.cc:482] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:16.920830 10572 raft_consensus.cc:3037] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:16.922994 10572 raft_consensus.cc:504] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.923485 10572 leader_election.cc:288] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [CANDIDATE]: Term 1 election: Requested vote from peers 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495), 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895)
I20260810 18:49:16.923946 9861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e507ad65d8e40dfa2918d526f74489f" candidate_uuid: "69e1e6de0bd9433aac8cde7a713a6337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52f3cf7bd132414eac2906a80eeb9ed7"
I20260810 18:49:16.923974 10092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e507ad65d8e40dfa2918d526f74489f" candidate_uuid: "69e1e6de0bd9433aac8cde7a713a6337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47c487ac956942a191e7b1aabc41d3da"
I20260810 18:49:16.924115 10092 raft_consensus.cc:3037] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:16.924124 9861 raft_consensus.cc:3037] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:16.926014 9861 raft_consensus.cc:2454] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69e1e6de0bd9433aac8cde7a713a6337 in term 1.
I20260810 18:49:16.926363 10092 raft_consensus.cc:2454] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69e1e6de0bd9433aac8cde7a713a6337 in term 1.
I20260810 18:49:16.926401 10211 leader_election.cc:302] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52f3cf7bd132414eac2906a80eeb9ed7, 69e1e6de0bd9433aac8cde7a713a6337; no voters:
I20260810 18:49:16.926664 10572 raft_consensus.cc:2781] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:16.927039 10572 raft_consensus.cc:686] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 LEADER]: Becoming Leader. State: Replica: 69e1e6de0bd9433aac8cde7a713a6337, State: Running, Role: LEADER
I20260810 18:49:16.927409 10572 consensus_queue.cc:237] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } }
I20260810 18:49:16.927994 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:16.927582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:55455)
0810 18:49:16.927771 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:16.927857 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 18:49:16.930936 9650 catalog_manager.cc:5495] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "69e1e6de0bd9433aac8cde7a713a6337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69e1e6de0bd9433aac8cde7a713a6337" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34889 } health_report { overall_health: HEALTHY } } }
I20260810 18:49:17.011495 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.006207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.29.174.195:34889)
0810 18:49:17.006416 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:17.006421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:17.006744 (+ 323us) server_negotiation.cc:407] Connection header received
0810 18:49:17.006922 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:17.006926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:17.007003 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:17.007121 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:17.007439 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.008132 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.009482 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.009961 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.010057 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:17.010466 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:17.010998 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:17.011101 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:17.011214 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 18:49:17.011284 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:49:17.015338 9861 raft_consensus.cc:1261] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Refusing update from remote peer 69e1e6de0bd9433aac8cde7a713a6337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:17.015604 10092 raft_consensus.cc:1261] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Refusing update from remote peer 69e1e6de0bd9433aac8cde7a713a6337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:17.016000 10576 consensus_queue.cc:1035] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52f3cf7bd132414eac2906a80eeb9ed7" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:17.016368 10572 consensus_queue.cc:1035] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47c487ac956942a191e7b1aabc41d3da" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 32895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:17.019675 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.019175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:44451)
0810 18:49:17.019343 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.019472 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 18:49:17.022706 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.019289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:41471)
0810 18:49:17.022383 (+ 3094us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.022514 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2936}
I20260810 18:49:17.028641 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.028173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.193:51535)
0810 18:49:17.028390 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.028478 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260810 18:49:17.029925 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.029434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.195:47041)
0810 18:49:17.029662 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.029749 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 18:49:17.030222 10581 mvcc.cc:205] Tried to move back new op lower bound from 7317044252728045568 to 7317044252378779648. Current Snapshot: MvccSnapshot[applied={T|T < 7317044252728045568 or (T in {7317044252728045568})}]
I20260810 18:49:17.031884 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.030961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.29.174.194:33555)
0810 18:49:17.031579 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.031688 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305}
I20260810 18:49:17.045047 9650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:17.045316 9650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:17.050617 9650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P defc7503518a41b0a103c6be9fc8b4ec: Sending DeleteTablet for 3 replicas of tablet 4e507ad65d8e40dfa2918d526f74489f
I20260810 18:49:17.051522 9830 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e507ad65d8e40dfa2918d526f74489f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:17 UTC) from {username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:57728
I20260810 18:49:17.051704 10287 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e507ad65d8e40dfa2918d526f74489f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:17 UTC) from {username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:35512
I20260810 18:49:17.052112 10060 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e507ad65d8e40dfa2918d526f74489f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:17 UTC) from {username='oryx', principal='oryx/127.29.174.252@KRBTEST.COM'} at 127.0.0.1:33644
I20260810 18:49:17.052768 10601 tablet_replica.cc:330] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: stopping tablet replica
I20260810 18:49:17.053009 10601 raft_consensus.cc:2229] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:17.053063 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 9609
I20260810 18:49:17.053419 10601 raft_consensus.cc:2258] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:17.055496 10601 ts_tablet_manager.cc:1905] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:17.062369 10601 ts_tablet_manager.cc:1918] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:17.062559 10601 log.cc:1196] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/4e507ad65d8e40dfa2918d526f74489f
I20260810 18:49:17.063210 10601 ts_tablet_manager.cc:1939] T 4e507ad65d8e40dfa2918d526f74489f P 47c487ac956942a191e7b1aabc41d3da: Deleting consensus metadata
I20260810 18:49:17.063269 10599 tablet_replica.cc:330] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: stopping tablet replica
I20260810 18:49:17.063434 10599 raft_consensus.cc:2229] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:17.063728 10599 raft_consensus.cc:2258] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:17.064438 9629 catalog_manager.cc:4841] TS 47c487ac956942a191e7b1aabc41d3da (127.29.174.194:32895): tablet 4e507ad65d8e40dfa2918d526f74489f (table test-table [id=41b8f471a6b04614a053722bb659a753]) successfully deleted
I20260810 18:49:17.064822 10599 ts_tablet_manager.cc:1905] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:17.069396 10600 tablet_replica.cc:330] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: stopping tablet replica
I20260810 18:49:17.069576 10600 raft_consensus.cc:2229] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:17.070200 10599 ts_tablet_manager.cc:1918] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:17.070175 10600 raft_consensus.cc:2258] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:17.070328 10599 log.cc:1196] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/4e507ad65d8e40dfa2918d526f74489f
I20260810 18:49:17.070803 10599 ts_tablet_manager.cc:1939] T 4e507ad65d8e40dfa2918d526f74489f P 52f3cf7bd132414eac2906a80eeb9ed7: Deleting consensus metadata
I20260810 18:49:17.071833 9629 catalog_manager.cc:4841] TS 52f3cf7bd132414eac2906a80eeb9ed7 (127.29.174.193:39495): tablet 4e507ad65d8e40dfa2918d526f74489f (table test-table [id=41b8f471a6b04614a053722bb659a753]) successfully deleted
I20260810 18:49:17.072407 10600 ts_tablet_manager.cc:1905] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:17.077728 10600 ts_tablet_manager.cc:1918] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:17.077872 10600 log.cc:1196] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/4e507ad65d8e40dfa2918d526f74489f
I20260810 18:49:17.078398 10600 ts_tablet_manager.cc:1939] T 4e507ad65d8e40dfa2918d526f74489f P 69e1e6de0bd9433aac8cde7a713a6337: Deleting consensus metadata
I20260810 18:49:17.079555 9631 catalog_manager.cc:4841] TS 69e1e6de0bd9433aac8cde7a713a6337 (127.29.174.195:34889): tablet 4e507ad65d8e40dfa2918d526f74489f (table test-table [id=41b8f471a6b04614a053722bb659a753]) successfully deleted
I20260810 18:49:17.083429 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.054416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.29.174.193:39495)
0810 18:49:17.054586 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:17.054592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:17.066799 (+ 12207us) server_negotiation.cc:407] Connection header received
0810 18:49:17.067028 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:17.067033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:17.067132 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:17.067218 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:17.070707 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.071810 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.073682 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.074259 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.074417 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:17.077723 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:17.077742 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:17.077761 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:17.077798 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:17.080803 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.081546 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.081552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.081554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.081682 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.082078 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.082082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.082084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.082508 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:17.082735 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:17.083024 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 18:49:17.083173 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:49:17.084014 9922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:17.139226 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.138375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57246)
0810 18:49:17.138867 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.138990 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":236,"threads_started":1}
I20260810 18:49:17.140734 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.140317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57250)
0810 18:49:17.140479 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.140588 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:49:17.182160 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 9609
I20260810 18:49:17.211102 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 9947
I20260810 18:49:17.224474 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.211818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.29.174.194:32895)
0810 18:49:17.211990 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:17.211997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:17.212054 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:49:17.212257 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:17.212262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:17.212328 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:17.212412 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:17.213722 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.214589 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.215779 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.216174 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.216293 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:17.218729 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:17.218745 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:17.218762 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:17.218803 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:17.221511 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.222120 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.222128 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.222130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.222273 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.222656 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.222660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.222662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.223018 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:17.223357 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:17.223954 (+ 597us) server_negotiation.cc:299] Negotiation successful
0810 18:49:17.224174 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:49:17.225147 10153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:17.367498 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 9947
I20260810 18:49:17.394595 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 10180
I20260810 18:49:17.408366 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.395325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35530 (local address 127.29.174.195:34889)
0810 18:49:17.395504 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:17.395510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:17.395593 (+ 83us) server_negotiation.cc:407] Connection header received
0810 18:49:17.395777 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:17.395781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:17.395873 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:17.395965 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:17.397306 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.398103 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.399435 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:17.399895 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:17.400011 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:17.402484 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:17.402497 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:17.402516 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:17.402553 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:17.405717 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.406331 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.406363 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.406367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.406534 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:17.406883 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:17.406888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:17.406891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:17.407247 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:17.407552 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:17.407826 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 18:49:17.408036 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:49:17.409101 10387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:17.536190 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 10180
I20260810 18:49:17.563700 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 9124
I20260810 18:49:17.564734 9399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:17.579175 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.578780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57266)
0810 18:49:17.578955 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.579038 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:49:17.580127 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:17.579843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:37617 (local address 127.0.0.1:57280)
0810 18:49:17.579952 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:17.580017 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:37617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 18:49:17.683876 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 9124
I20260810 18:49:17.707612 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 9289
I20260810 18:49:17.708603 9692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:17.825891 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 9289
2026-08-10T18:49:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10205 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:17 dist-test-slave-6zbq krb5kdc[10617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:17 dist-test-slave-6zbq krb5kdc[10617](info): set up 2 sockets
Aug 10 18:49:17 dist-test-slave-6zbq krb5kdc[10617](info): commencing operation
krb5kdc: starting...
W20260810 18:49:19.063273 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:19 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab.
Entry for principal oryx/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab.
I20260810 18:49:19.302088 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43437
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab
--principal=oryx/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:43437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:19.628082 10635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:19.628374 10635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:19.628515 10635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:19.637030 10635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:19.637135 10635 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:49:19.637198 10635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:19.637280 10635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:19.637347 10635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:19.637425 10635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:19.647004 10635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:43437
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.254.keytab
--principal=oryx/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:19.648166 10635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:19.649402 10635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:19.660590 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:19.660636 10653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:19.660677 10649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:19.661489 10635 server_base.cc:1034] running on GCE node
I20260810 18:49:19.662168 10635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:19.663455 10635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:19.664664 10635 hybrid_clock.cc:648] HybridClock initialized: now 1786387759664592 us; error 44 us; skew 500 ppm
I20260810 18:49:19.664925 10635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:19 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387759, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:19.667770 10635 init.cc:377] Logged in from keytab as oryx/127.29.174.254@KRBTEST.COM (short username oryx)
I20260810 18:49:19.669031 10635 webserver.cc:460] Webserver started at http://127.29.174.254:36873/ using document root <none> and password file <none>
I20260810 18:49:19.669656 10635 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:19.669754 10635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:19.670048 10635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:19.672833 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "f4c3defda7ac4df99a2e4b35b13c36de"
format_stamp: "Formatted at 2026-08-10 18:49:19 on dist-test-slave-6zbq"
server_key: "edbfdad25a410f89f931853a7e55a343"
server_key_iv: "4c1651a21eedba7ad23ebc36b497b917"
server_key_version: "encryptionkey@0"
I20260810 18:49:19.673590 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "f4c3defda7ac4df99a2e4b35b13c36de"
format_stamp: "Formatted at 2026-08-10 18:49:19 on dist-test-slave-6zbq"
server_key: "edbfdad25a410f89f931853a7e55a343"
server_key_iv: "4c1651a21eedba7ad23ebc36b497b917"
server_key_version: "encryptionkey@0"
I20260810 18:49:19.678882 10635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:49:19.682245 10665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:19.683485 10635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:49:19.683640 10635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "f4c3defda7ac4df99a2e4b35b13c36de"
format_stamp: "Formatted at 2026-08-10 18:49:19 on dist-test-slave-6zbq"
server_key: "edbfdad25a410f89f931853a7e55a343"
server_key_iv: "4c1651a21eedba7ad23ebc36b497b917"
server_key_version: "encryptionkey@0"
I20260810 18:49:19.683835 10635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:19.705706 10635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:19.709707 10635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:19.709949 10635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:19.729552 10635 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:43437
I20260810 18:49:19.729598 10756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:43437 every 8 connection(s)
I20260810 18:49:19.730926 10635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:19.734460 10757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:19.735715 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 10635
I20260810 18:49:19.735901 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:19.736507 30395 external_mini_cluster.cc:1428] Setting key c795f0f8706b25a3d31baf10547f8969
I20260810 18:49:19.745623 10757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Bootstrap starting.
I20260810 18:49:19.749148 10757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:19.750308 10757 log.cc:824] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:49:19 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:19.752862 10757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: No bootstrap required, opened a new log
I20260810 18:49:19.758180 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:19.738571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.29.174.254:43437)
0810 18:49:19.739384 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:19.739397 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:19.739439 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:49:19.741143 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:19.741164 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:19.741502 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:19.741794 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:19.743492 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:19.744757 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:19.746392 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:19.747033 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:19.747163 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:19.751563 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:19.751571 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:19.751583 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:19.751617 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:19.754498 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:19.755115 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:19.755121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:19.755127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:19.755300 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:19.755663 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:19.755667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:19.755669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:19.756158 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:19.756448 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:19.756876 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 18:49:19.757256 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":366,"threads_started":1}
I20260810 18:49:19.760609 10757 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } }
I20260810 18:49:19.761023 10757 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:19.761149 10757 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4c3defda7ac4df99a2e4b35b13c36de, State: Initialized, Role: FOLLOWER
I20260810 18:49:19.761636 10757 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } }
I20260810 18:49:19.761852 10757 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:19.761945 10757 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:19.762071 10757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:19.764398 10757 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } }
I20260810 18:49:19.764835 10757 leader_election.cc:302] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4c3defda7ac4df99a2e4b35b13c36de; no voters:
I20260810 18:49:19.765473 10757 leader_election.cc:288] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:19.765623 10764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:19.766381 10764 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [term 1 LEADER]: Becoming Leader. State: Replica: f4c3defda7ac4df99a2e4b35b13c36de, State: Running, Role: LEADER
I20260810 18:49:19.766950 10764 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } }
I20260810 18:49:19.767472 10757 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:19.769408 10767 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4c3defda7ac4df99a2e4b35b13c36de. Latest consensus state: current_term: 1 leader_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } } }
I20260810 18:49:19.769872 10767 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:19.769421 10765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4c3defda7ac4df99a2e4b35b13c36de" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43437 } } }
I20260810 18:49:19.771365 10765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:19.772814 10772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:19.781607 10772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:19.789635 10772 catalog_manager.cc:1347] Generated new cluster ID: 0018c15dc5b643d1953129eef05bdf4f
I20260810 18:49:19.789738 10772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:19.850409 10772 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:19.851758 10772 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:19.864980 10772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Generated new TSK 0
I20260810 18:49:19.865787 10772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
I20260810 18:49:19.934175 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43437
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:20.331254 10800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:20.331501 10800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:20.331635 10800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:20.338523 10800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:20.338615 10800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:20.338744 10800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:49:20.338838 10800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:20.348114 10800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:20.349242 10800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:20.350473 10800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:20.362308 10812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:20.362330 10814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:20.362916 10808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:20.363468 10800 server_base.cc:1034] running on GCE node
I20260810 18:49:20.364082 10800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:20.364972 10800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:20.366220 10800 hybrid_clock.cc:648] HybridClock initialized: now 1786387760366145 us; error 54 us; skew 500 ppm
I20260810 18:49:20.366536 10800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:20 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387760, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:20.369521 10800 init.cc:377] Logged in from keytab as oryx/127.29.174.193@KRBTEST.COM (short username oryx)
I20260810 18:49:20.370890 10800 webserver.cc:460] Webserver started at http://127.29.174.193:32919/ using document root <none> and password file <none>
I20260810 18:49:20.371526 10800 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:20.371623 10800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:20.371915 10800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:20.374714 10800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "4344a0c5741459bde244d6d5e7c35522"
server_key_iv: "72eec5fe9fa88b9448a25800d160da61"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.375461 10800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "4344a0c5741459bde244d6d5e7c35522"
server_key_iv: "72eec5fe9fa88b9448a25800d160da61"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.380098 10800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:20.383301 10823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:20.384438 10800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:49:20.384584 10800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "4344a0c5741459bde244d6d5e7c35522"
server_key_iv: "72eec5fe9fa88b9448a25800d160da61"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.384785 10800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:20.407433 10800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:20.411104 10800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:20.411360 10800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:20.413854 10800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:20.413954 10800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:20.414139 10800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:20.414216 10800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:49:20 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387760, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:20.434578 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.414233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49047 (local address 127.29.174.254:43437)
0810 18:49:20.414714 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:20.414721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:20.415799 (+ 1078us) server_negotiation.cc:407] Connection header received
0810 18:49:20.417984 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:20.417989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:20.418072 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:20.418165 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:20.420268 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.421398 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.423683 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.424322 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.424458 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.428841 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:20.428846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:20.428849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:20.428881 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:20.431513 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.432389 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.432395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.432399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.432524 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.432912 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.432918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.432921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.433180 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:20.433313 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:20.433988 (+ 675us) server_negotiation.cc:299] Negotiation successful
0810 18:49:20.434207 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":259,"threads_started":1}
I20260810 18:49:20.435827 10839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.414906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.193:49047)
0810 18:49:20.415641 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:20.415679 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:20.417683 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:20.418468 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:20.418480 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:20.419152 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:20.420028 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.420053 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.421586 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.421591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.423531 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.423544 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.424570 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.424579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.424796 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.425445 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:20.425470 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:20.428606 (+ 3136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:20.431658 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.431667 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.431681 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.432268 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.432624 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.432629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.432633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.432799 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.433305 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:20.433312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:20.433629 (+ 317us) client_negotiation.cc:769] Sending connection context
0810 18:49:20.433846 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 18:49:20.434333 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":323,"threads_started":1}
I20260810 18:49:20.461707 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.448379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48335 (local address 127.29.174.254:43437)
0810 18:49:20.448567 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:20.448573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:20.448941 (+ 368us) server_negotiation.cc:407] Connection header received
0810 18:49:20.449110 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:20.449116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:20.449191 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:20.449277 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:20.450702 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.451537 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.453074 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.453828 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.453989 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.456017 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:20.456024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:20.456029 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:20.456065 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:20.459029 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.459777 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.459783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.459786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.459935 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.460409 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.460415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.460418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.460689 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:20.460849 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:20.461308 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 18:49:20.461469 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:49:20.462030 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.447928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.193:48335)
0810 18:49:20.448836 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:20.448857 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:20.448997 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:20.449317 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:20.449323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:20.450169 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:20.450564 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.450577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.451716 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.451722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.452928 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.452942 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.454141 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.454150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.454319 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.455011 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:20.455028 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:20.455816 (+ 788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:20.459180 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.459186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.459190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.459654 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.460086 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.460091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.460095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.460278 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.460809 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:20.460815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:20.461036 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 18:49:20.461200 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:49:20.461519 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":604,"threads_started":1}
I20260810 18:49:20.463027 10800 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:44395
I20260810 18:49:20.463089 11010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:44395 every 8 connection(s)
I20260810 18:49:20.464551 10800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:20.466125 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 10800
I20260810 18:49:20.466317 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:20.466972 30395 external_mini_cluster.cc:1428] Setting key 696e8aef5e3e7397c86efcffcde97f08
I20260810 18:49:20.466964 11012 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43437
I20260810 18:49:20.467202 11012 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:20.468009 11012 heartbeater.cc:511] Master 127.29.174.254:43437 requested a full tablet report, sending...
I20260810 18:49:20.470201 10694 ts_manager.cc:194] Registered new tserver with Master: 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395)
I20260810 18:49:20.472090 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:49047
WARNING: no policy specified for oryx/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
I20260810 18:49:20.519623 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43437
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:20.846236 11018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:20.846514 11018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:20.846611 11018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:20.853479 11018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:20.853576 11018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:20.853665 11018 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:49:20.853777 11018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:20.863924 11018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:20.865178 11018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:20.866482 11018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:20.879146 11025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:20.879101 11026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:20.879155 11029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:20.879850 11018 server_base.cc:1034] running on GCE node
I20260810 18:49:20.880442 11018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:20.881292 11018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:20.882485 11018 hybrid_clock.cc:648] HybridClock initialized: now 1786387760882428 us; error 29 us; skew 500 ppm
I20260810 18:49:20.882802 11018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:20 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387760, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:20.885646 11018 init.cc:377] Logged in from keytab as oryx/127.29.174.194@KRBTEST.COM (short username oryx)
I20260810 18:49:20.886988 11018 webserver.cc:460] Webserver started at http://127.29.174.194:34041/ using document root <none> and password file <none>
I20260810 18:49:20.887666 11018 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:20.887759 11018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:20.888084 11018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:20.890910 11018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "a4e60ef0d8874e8e80b8775451cabfc2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "3d6e22643236e640a1299dccc9d5305b"
server_key_iv: "f280ef5d2c3bd0d9169daca32ee43215"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.891860 11018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "a4e60ef0d8874e8e80b8775451cabfc2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "3d6e22643236e640a1299dccc9d5305b"
server_key_iv: "f280ef5d2c3bd0d9169daca32ee43215"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.896706 11018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:49:20.899909 11039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:20.901189 11018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:49:20.901342 11018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "a4e60ef0d8874e8e80b8775451cabfc2"
format_stamp: "Formatted at 2026-08-10 18:49:20 on dist-test-slave-6zbq"
server_key: "3d6e22643236e640a1299dccc9d5305b"
server_key_iv: "f280ef5d2c3bd0d9169daca32ee43215"
server_key_version: "encryptionkey@0"
I20260810 18:49:20.901535 11018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:20.935460 11018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:20.939263 11018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:20.939489 11018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:20.941980 11018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:20.942063 11018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:20.942188 11018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:20.942255 11018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:49:20 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387760, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:20.963368 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.942621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:37177 (local address 127.29.174.254:43437)
0810 18:49:20.942815 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:20.942822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:20.944233 (+ 1411us) server_negotiation.cc:407] Connection header received
0810 18:49:20.946621 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:20.946627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:20.946725 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:20.946850 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:20.948899 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.950020 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.952012 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.952650 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.952792 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.957008 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:20.957014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:20.957017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:20.957048 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:20.959973 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.960799 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.960805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.960809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.960949 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.961556 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.961562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.961565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.961868 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:20.962002 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:20.962804 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 18:49:20.963020 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:49:20.964619 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.943170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.194:37177)
0810 18:49:20.944043 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:20.944093 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:20.946207 (+ 2114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:20.947141 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:20.947153 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:20.947871 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:20.948683 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.948709 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.950219 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.950225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.951860 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.951876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.952908 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.952917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.953117 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.953861 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:20.953894 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:20.956814 (+ 2920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:20.960109 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.960116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.960127 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.960663 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.961098 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.961104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.961111 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.961419 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.962003 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:20.962011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:20.962393 (+ 382us) client_negotiation.cc:769] Sending connection context
0810 18:49:20.962678 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 18:49:20.963181 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":331,"threads_started":1}
I20260810 18:49:20.983510 11018 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:33333
I20260810 18:49:20.983551 11241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:33333 every 8 connection(s)
I20260810 18:49:20.984725 11018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:20.985191 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.971326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:51895 (local address 127.29.174.254:43437)
0810 18:49:20.971507 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:20.971512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:20.972199 (+ 687us) server_negotiation.cc:407] Connection header received
0810 18:49:20.972322 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:20.972328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:20.972409 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:20.972512 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:20.973293 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.974508 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.976770 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.977308 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.977431 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.979636 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:20.979644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:20.979649 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:20.979686 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:20.982630 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.983352 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.983355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.983358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.983467 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:20.983816 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:20.983821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:20.983825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:20.984138 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:20.984265 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:20.984661 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 18:49:20.984868 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:49:20.985322 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:20.971433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.194:51895)
0810 18:49:20.972099 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:20.972116 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:20.972225 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:20.972514 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:20.972517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:20.972904 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:20.973183 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.973191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.974681 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.974688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.976610 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:20.976631 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:20.977537 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:20.977544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:20.977727 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:20.978419 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:20.978443 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:20.979416 (+ 973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:20.982802 (+ 3386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.982807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.982811 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.983223 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.983579 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:20.983584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:20.983588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:20.983723 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:20.984258 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:20.984263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:20.984430 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 18:49:20.984552 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 18:49:20.984852 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":393,"threads_started":1}
I20260810 18:49:20.987236 11242 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43437
I20260810 18:49:20.987488 11242 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:20.988296 11242 heartbeater.cc:511] Master 127.29.174.254:43437 requested a full tablet report, sending...
I20260810 18:49:20.989594 10694 ts_manager.cc:194] Registered new tserver with Master: a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333)
I20260810 18:49:20.990365 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:37177
I20260810 18:49:20.992264 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 11018
I20260810 18:49:20.992415 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:20.992823 30395 external_mini_cluster.cc:1428] Setting key 1744084e181ccc6a8b03b7e6e3ff1a71
WARNING: no policy specified for oryx/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
I20260810 18:49:21.040982 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43437
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:21.349382 11249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:21.349628 11249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:21.349754 11249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:21.356635 11249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:21.356738 11249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:21.356859 11249 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:49:21.356961 11249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:21.366097 11249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.174.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:21.367295 11249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:21.368539 11249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:21.380540 11261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:21.380623 11260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:21.380649 11263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:21.381183 11249 server_base.cc:1034] running on GCE node
I20260810 18:49:21.381721 11249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:21.382550 11249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:21.383761 11249 hybrid_clock.cc:648] HybridClock initialized: now 1786387761383702 us; error 25 us; skew 500 ppm
I20260810 18:49:21.384003 11249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387761, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:21.386893 11249 init.cc:377] Logged in from keytab as oryx/127.29.174.195@KRBTEST.COM (short username oryx)
I20260810 18:49:21.388167 11249 webserver.cc:460] Webserver started at http://127.29.174.195:36281/ using document root <none> and password file <none>
I20260810 18:49:21.388782 11249 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:21.388883 11249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:21.389170 11249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:21.391858 11249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
format_stamp: "Formatted at 2026-08-10 18:49:21 on dist-test-slave-6zbq"
server_key: "469a71280be02f19a26c3ad2c22da7b5"
server_key_iv: "b8e264d90aa5b207053d487464560495"
server_key_version: "encryptionkey@0"
I20260810 18:49:21.392596 11249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
format_stamp: "Formatted at 2026-08-10 18:49:21 on dist-test-slave-6zbq"
server_key: "469a71280be02f19a26c3ad2c22da7b5"
server_key_iv: "b8e264d90aa5b207053d487464560495"
server_key_version: "encryptionkey@0"
I20260810 18:49:21.397267 11249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:49:21.400395 11271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:21.401558 11249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:21.401695 11249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
format_stamp: "Formatted at 2026-08-10 18:49:21 on dist-test-slave-6zbq"
server_key: "469a71280be02f19a26c3ad2c22da7b5"
server_key_iv: "b8e264d90aa5b207053d487464560495"
server_key_version: "encryptionkey@0"
I20260810 18:49:21.401898 11249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:21.416958 11249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:21.420555 11249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:21.420789 11249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:21.423328 11249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:21.423409 11249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:21.423589 11249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:21.423666 11249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387761, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:21.445475 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.423741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35483 (local address 127.29.174.254:43437)
0810 18:49:21.423937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.423945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.425315 (+ 1370us) server_negotiation.cc:407] Connection header received
0810 18:49:21.428242 (+ 2927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.428249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.428340 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.428445 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:21.430531 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.431673 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.433863 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.434426 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.434538 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.438753 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:21.438761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:21.438767 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:21.438811 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:21.442045 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.443007 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.443012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.443016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.443185 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.443760 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.443763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.443765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.444083 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:21.444233 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.445057 (+ 824us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.445218 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:49:21.447019 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.424281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.195:35483)
0810 18:49:21.425128 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.425191 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.427866 (+ 2675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.428772 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.428787 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.429532 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:21.430275 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.430306 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.431830 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.431836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.433715 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.433728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.434666 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.434674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.434860 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.435487 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:21.435519 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:21.438501 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:21.442191 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.442199 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.442214 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.442837 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.443292 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.443298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.443301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.443595 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.444242 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:21.444252 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:21.444668 (+ 416us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.444959 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.445469 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":330,"threads_started":1}
I20260810 18:49:21.464486 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.453024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:57387 (local address 127.29.174.254:43437)
0810 18:49:21.453181 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.453187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.453797 (+ 610us) server_negotiation.cc:407] Connection header received
0810 18:49:21.453935 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.453940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.454012 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.454090 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:21.455194 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.456118 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.457472 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.457894 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.458009 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.459585 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:21.459590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:21.459593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:21.459621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:21.461926 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.462706 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.462712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.462715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.462848 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.463193 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.463197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.463199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.463444 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:21.463580 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.463967 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.464165 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:49:21.464555 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.453175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.29.174.195:57387)
0810 18:49:21.453704 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.453719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.453836 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.454135 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.454140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.454602 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:21.455050 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.455064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.456271 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.456275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.457336 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.457351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.458081 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.458087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.458223 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.458747 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:21.458763 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:21.459408 (+ 645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:21.462064 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.462070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.462074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.462553 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.462967 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.462970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.462973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.463080 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.463556 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:21.463561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:21.463725 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.463833 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.464128 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":279,"threads_started":1}
I20260810 18:49:21.474715 11012 heartbeater.cc:503] Master 127.29.174.254:43437 was elected leader, sending a full tablet report...
I20260810 18:49:21.474778 11249 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:42183
I20260810 18:49:21.474790 11477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:42183 every 8 connection(s)
I20260810 18:49:21.475842 11249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:21.477638 11479 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43437
I20260810 18:49:21.477833 11479 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:21.478447 11479 heartbeater.cc:511] Master 127.29.174.254:43437 requested a full tablet report, sending...
I20260810 18:49:21.479586 10694 ts_manager.cc:194] Registered new tserver with Master: cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:21.480456 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 11249
I20260810 18:49:21.480475 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:35483
I20260810 18:49:21.480628 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:21.481091 30395 external_mini_cluster.cc:1428] Setting key 6cb05b0221ca0533884610f8e8078d9f
I20260810 18:49:21.483668 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387761, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:49:21.540277 10836 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:21.526856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.29.174.254:43437)
0810 18:49:21.527077 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.527084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.527522 (+ 438us) server_negotiation.cc:407] Connection header received
0810 18:49:21.527662 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.527666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.527739 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.527827 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:21.529041 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.529778 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.530973 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.531366 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.531456 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.539970 (+ 8514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59468: BlockingRecv error: recv got EOF from 127.0.0.1:59468 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387761, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:21.556231 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.543783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59474 (local address 127.29.174.254:43437)
0810 18:49:21.543968 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.543974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.544188 (+ 214us) server_negotiation.cc:407] Connection header received
0810 18:49:21.544354 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.544359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.544433 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.544511 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:21.545546 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.546240 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.547485 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.547915 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.548019 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.551611 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:21.551617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:21.551620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:21.551677 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:21.554072 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.554713 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.554717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.554720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.554846 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.555123 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.555126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.555128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.555384 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:21.555594 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.555778 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.555950 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:49:21.559888 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:21.566504 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:21.587330 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.577856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.29.174.194:33333)
0810 18:49:21.578637 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.578646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.578676 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:49:21.579015 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.579023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.579271 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.579447 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.580429 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.582040 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.584878 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.585921 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.586108 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.586277 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.586882 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.587054 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":1195,"threads_started":1}
I20260810 18:49:21.587541 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.577697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:33333 (local address 127.0.0.1:38106)
0810 18:49:21.578505 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.578543 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.578724 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.579693 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.579701 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.579732 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.580201 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.580216 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.582206 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.582214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.584732 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.584744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.586440 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.586447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.586633 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.586686 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.586822 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.586997 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":4,"thread_start_us":339,"threads_started":1}
I20260810 18:49:21.588238 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.578851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.0.0.1:59334)
0810 18:49:21.581170 (+ 2319us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.581186 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.581306 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.581820 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.581824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.581848 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.582225 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.582239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.583673 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.583678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.585375 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.585389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.587451 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.587457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.587635 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.587668 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.587756 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.587877 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2205,"mutex_wait_us":5,"spinlock_wait_cycles":28160,"thread_start_us":1975,"threads_started":1}
I20260810 18:49:21.588891 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.579294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59334 (local address 127.29.174.193:44395)
0810 18:49:21.580755 (+ 1461us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.580764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.581412 (+ 648us) server_negotiation.cc:407] Connection header received
0810 18:49:21.581498 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.581503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.581654 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.581802 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.582429 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.583552 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.585564 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.586986 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.587146 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.587304 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.588442 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.588611 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1326,"thread_start_us":1213,"threads_started":1}
I20260810 18:49:21.589587 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.578294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:42183 (local address 127.0.0.1:36338)
0810 18:49:21.581143 (+ 2849us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.581164 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.581296 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.582116 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.582124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.582153 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.582670 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.582683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.584636 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.584642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.586602 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.586614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.588834 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.588839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.589015 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.589046 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.589132 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.589259 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2677,"spinlock_wait_cycles":3200,"thread_start_us":2660,"threads_started":1}
I20260810 18:49:21.589710 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.578718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.29.174.195:42183)
0810 18:49:21.580347 (+ 1629us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.580356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.581266 (+ 910us) server_negotiation.cc:407] Connection header received
0810 18:49:21.581436 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.581444 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.581641 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.581805 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.582918 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.584496 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.586742 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.588095 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.588273 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.588428 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.589216 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.589386 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1443,"thread_start_us":1218,"threads_started":1}
I20260810 18:49:21.591364 11124 tablet_service.cc:1467] Processing CreateTablet for tablet a09dd1da783e492bb00a3f65c66d2052 (DEFAULT_TABLE table=test-table [id=ccacd70450034f9297b4dcb0dc44e739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.591827 10903 tablet_service.cc:1467] Processing CreateTablet for tablet a09dd1da783e492bb00a3f65c66d2052 (DEFAULT_TABLE table=test-table [id=ccacd70450034f9297b4dcb0dc44e739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.592478 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a09dd1da783e492bb00a3f65c66d2052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.592767 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a09dd1da783e492bb00a3f65c66d2052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.597373 11364 tablet_service.cc:1467] Processing CreateTablet for tablet a09dd1da783e492bb00a3f65c66d2052 (DEFAULT_TABLE table=test-table [id=ccacd70450034f9297b4dcb0dc44e739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.598764 11364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a09dd1da783e492bb00a3f65c66d2052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.604223 11523 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Bootstrap starting.
I20260810 18:49:21.607373 11523 tablet_bootstrap.cc:654] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.608379 11523 log.cc:824] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:21.612298 11525 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Bootstrap starting.
I20260810 18:49:21.615041 11529 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Bootstrap starting.
I20260810 18:49:21.616345 11525 tablet_bootstrap.cc:654] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.617601 11525 log.cc:824] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:21.618194 11529 tablet_bootstrap.cc:654] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.618523 11523 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: No bootstrap required, opened a new log
I20260810 18:49:21.618768 11523 ts_tablet_manager.cc:1397] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260810 18:49:21.619292 11529 log.cc:824] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:21.620563 11525 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: No bootstrap required, opened a new log
I20260810 18:49:21.620806 11525 ts_tablet_manager.cc:1397] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 18:49:21.621552 11529 tablet_bootstrap.cc:492] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: No bootstrap required, opened a new log
I20260810 18:49:21.621759 11529 ts_tablet_manager.cc:1397] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:49:21.628757 11529 raft_consensus.cc:348] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.629074 11529 raft_consensus.cc:374] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.628836 11523 raft_consensus.cc:348] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.629166 11529 raft_consensus.cc:729] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.629240 11523 raft_consensus.cc:374] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.629338 11523 raft_consensus.cc:729] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.629722 11529 consensus_queue.cc:260] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.629894 11523 consensus_queue.cc:260] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.630836 11479 heartbeater.cc:503] Master 127.29.174.254:43437 was elected leader, sending a full tablet report...
I20260810 18:49:21.631253 11529 ts_tablet_manager.cc:1428] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260810 18:49:21.631706 11525 raft_consensus.cc:348] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.632018 11525 raft_consensus.cc:374] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.632105 11525 raft_consensus.cc:729] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.632606 11525 consensus_queue.cc:260] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.634268 11525 ts_tablet_manager.cc:1428] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260810 18:49:21.635133 11242 heartbeater.cc:503] Master 127.29.174.254:43437 was elected leader, sending a full tablet report...
I20260810 18:49:21.635728 11523 ts_tablet_manager.cc:1428] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260810 18:49:21.641105 11537 raft_consensus.cc:482] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:21.641360 11537 raft_consensus.cc:504] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.642980 11537 leader_election.cc:288] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183), 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395)
I20260810 18:49:21.649802 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.643198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:38151 (local address 127.29.174.195:42183)
0810 18:49:21.643392 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.643399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.643962 (+ 563us) server_negotiation.cc:407] Connection header received
0810 18:49:21.644102 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.644107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.644188 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.644301 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.644735 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.645864 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.648071 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.648891 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.649031 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.649105 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.649444 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.649565 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:49:21.650043 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.643174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:42183 (local address 127.29.174.194:38151)
0810 18:49:21.643860 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.643875 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.643984 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.644307 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.644310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.644334 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.644625 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.644633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.645969 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.645972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.647939 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.647950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.649119 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.649130 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.649300 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.649324 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.649407 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.649550 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":4,"thread_start_us":3354,"threads_started":1}
I20260810 18:49:21.650810 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09dd1da783e492bb00a3f65c66d2052" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" is_pre_election: true
I20260810 18:49:21.651252 11398 raft_consensus.cc:2454] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:21.651810 11042 leader_election.cc:302] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4e60ef0d8874e8e80b8775451cabfc2, cd1ebabf309e4c5283ae2fbafd40625a; no voters:
I20260810 18:49:21.651958 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.643434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:38421 (local address 127.29.174.193:44395)
0810 18:49:21.643633 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.643639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.646945 (+ 3306us) server_negotiation.cc:407] Connection header received
0810 18:49:21.647078 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.647082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.647143 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.647211 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.647822 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.648508 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.650313 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.651147 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.651292 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.651359 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.651624 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.651744 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:49:21.652112 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.643198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.29.174.194:38421)
0810 18:49:21.646836 (+ 3638us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.646862 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.646983 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.647245 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.647250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.647273 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.647682 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.647694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.648634 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.648638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.650168 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.650183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.651328 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.651335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.651486 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.651509 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.651585 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.651698 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3510,"mutex_wait_us":20,"thread_start_us":413,"threads_started":1}
I20260810 18:49:21.652211 11537 raft_consensus.cc:2781] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:21.652359 11537 raft_consensus.cc:482] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:21.652463 11537 raft_consensus.cc:3037] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.652901 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09dd1da783e492bb00a3f65c66d2052" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" is_pre_election: true
I20260810 18:49:21.653291 10940 raft_consensus.cc:2454] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:21.654744 11537 raft_consensus.cc:504] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.655284 11537 leader_election.cc:288] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Requested vote from peers cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183), 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395)
I20260810 18:49:21.655687 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09dd1da783e492bb00a3f65c66d2052" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
I20260810 18:49:21.655875 11398 raft_consensus.cc:3037] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.658182 11398 raft_consensus.cc:2454] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:21.658538 11042 leader_election.cc:302] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4e60ef0d8874e8e80b8775451cabfc2, cd1ebabf309e4c5283ae2fbafd40625a; no voters:
I20260810 18:49:21.658524 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a09dd1da783e492bb00a3f65c66d2052" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
I20260810 18:49:21.658694 10940 raft_consensus.cc:3037] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.658780 11537 raft_consensus.cc:2781] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:21.659224 11537 raft_consensus.cc:686] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Becoming Leader. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Running, Role: LEADER
I20260810 18:49:21.659674 11537 consensus_queue.cc:237] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } }
I20260810 18:49:21.661114 10940 raft_consensus.cc:2454] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:21.664142 10692 catalog_manager.cc:5495] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:21.686861 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.682137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.29.174.194:33333)
0810 18:49:21.682328 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.682334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.682383 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:49:21.682466 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.682469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.682536 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.682653 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:21.682985 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.683672 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.684918 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.685306 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.685410 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.685819 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:21.686273 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:21.686435 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.686539 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.686615 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:49:21.694015 10940 raft_consensus.cc:1261] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:21.694093 11398 raft_consensus.cc:1261] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:21.694914 11542 consensus_queue.cc:1035] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:21.695251 11537 consensus_queue.cc:1035] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:21.707427 11547 mvcc.cc:205] Tried to move back new op lower bound from 7317044271890702336 to 7317044271761760256. Current Snapshot: MvccSnapshot[applied={T|T < 7317044271890702336}]
I20260810 18:49:21.708228 11546 mvcc.cc:205] Tried to move back new op lower bound from 7317044271890702336 to 7317044271761760256. Current Snapshot: MvccSnapshot[applied={T|T < 7317044271890702336}]
I20260810 18:49:21.720038 10692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:21.720273 10692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:21.723359 10692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Sending DeleteTablet for 3 replicas of tablet a09dd1da783e492bb00a3f65c66d2052
I20260810 18:49:21.724345 10903 tablet_service.cc:1514] Processing DeleteTablet for tablet a09dd1da783e492bb00a3f65c66d2052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:21 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59334
I20260810 18:49:21.724342 11364 tablet_service.cc:1514] Processing DeleteTablet for tablet a09dd1da783e492bb00a3f65c66d2052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:21 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36338
I20260810 18:49:21.724427 11124 tablet_service.cc:1514] Processing DeleteTablet for tablet a09dd1da783e492bb00a3f65c66d2052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:21 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38106
I20260810 18:49:21.727931 10692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:21.728595 10692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:21.738714 11122 tablet_service.cc:1467] Processing CreateTablet for tablet f847cc9d9b2f458c9a461707d3dada67 (DEFAULT_TABLE table=test-table [id=4bf1fb84fa06435bb2beef8363fd6636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.738945 11359 tablet_service.cc:1467] Processing CreateTablet for tablet f847cc9d9b2f458c9a461707d3dada67 (DEFAULT_TABLE table=test-table [id=4bf1fb84fa06435bb2beef8363fd6636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.739346 11122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f847cc9d9b2f458c9a461707d3dada67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.739495 11359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f847cc9d9b2f458c9a461707d3dada67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.742074 10901 tablet_service.cc:1467] Processing CreateTablet for tablet f847cc9d9b2f458c9a461707d3dada67 (DEFAULT_TABLE table=test-table [id=4bf1fb84fa06435bb2beef8363fd6636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:21.742704 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f847cc9d9b2f458c9a461707d3dada67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.744576 11560 tablet_replica.cc:330] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: stopping tablet replica
I20260810 18:49:21.744985 11560 raft_consensus.cc:2229] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:21.745432 11560 raft_consensus.cc:2258] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:21.746222 11558 tablet_replica.cc:330] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: stopping tablet replica
I20260810 18:49:21.746371 11529 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Bootstrap starting.
I20260810 18:49:21.746623 11558 raft_consensus.cc:2229] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:21.747083 11558 raft_consensus.cc:2258] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:21.748021 11529 tablet_bootstrap.cc:654] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.748528 11560 ts_tablet_manager.cc:1905] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:21.754808 11560 ts_tablet_manager.cc:1918] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:21.754956 11560 log.cc:1196] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/a09dd1da783e492bb00a3f65c66d2052
I20260810 18:49:21.755440 11560 ts_tablet_manager.cc:1939] T a09dd1da783e492bb00a3f65c66d2052 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting consensus metadata
I20260810 18:49:21.757085 10666 catalog_manager.cc:4841] TS 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395): tablet a09dd1da783e492bb00a3f65c66d2052 (table test-table [id=ccacd70450034f9297b4dcb0dc44e739]) successfully deleted
I20260810 18:49:21.757543 11525 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Bootstrap starting.
I20260810 18:49:21.758833 11523 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Bootstrap starting.
I20260810 18:49:21.759117 11525 tablet_bootstrap.cc:654] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.759203 11558 ts_tablet_manager.cc:1905] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:21.761334 11523 tablet_bootstrap.cc:654] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.761842 11529 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: No bootstrap required, opened a new log
I20260810 18:49:21.762041 11529 ts_tablet_manager.cc:1397] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260810 18:49:21.762894 11529 raft_consensus.cc:348] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.763244 11529 raft_consensus.cc:374] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.763384 11529 raft_consensus.cc:729] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.763737 11529 consensus_queue.cc:260] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.764427 11561 tablet_replica.cc:330] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: stopping tablet replica
I20260810 18:49:21.764789 11561 raft_consensus.cc:2229] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:21.765364 11561 raft_consensus.cc:2258] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:21.766106 11558 ts_tablet_manager.cc:1918] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:21.766258 11558 log.cc:1196] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/a09dd1da783e492bb00a3f65c66d2052
I20260810 18:49:21.766299 11523 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: No bootstrap required, opened a new log
I20260810 18:49:21.766472 11523 ts_tablet_manager.cc:1397] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260810 18:49:21.766806 11529 ts_tablet_manager.cc:1428] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 18:49:21.766924 11558 ts_tablet_manager.cc:1939] T a09dd1da783e492bb00a3f65c66d2052 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting consensus metadata
I20260810 18:49:21.767256 11523 raft_consensus.cc:348] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.767472 11523 raft_consensus.cc:374] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.767583 11523 raft_consensus.cc:729] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.767853 11523 consensus_queue.cc:260] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.768734 11523 ts_tablet_manager.cc:1428] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:21.768886 10668 catalog_manager.cc:4841] TS cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183): tablet a09dd1da783e492bb00a3f65c66d2052 (table test-table [id=ccacd70450034f9297b4dcb0dc44e739]) successfully deleted
I20260810 18:49:21.770615 11525 tablet_bootstrap.cc:492] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: No bootstrap required, opened a new log
I20260810 18:49:21.770762 11525 ts_tablet_manager.cc:1397] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260810 18:49:21.771518 11525 raft_consensus.cc:348] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.771744 11525 raft_consensus.cc:374] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.771839 11525 raft_consensus.cc:729] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.771915 11561 ts_tablet_manager.cc:1905] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:21.772125 11525 consensus_queue.cc:260] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.772831 11525 ts_tablet_manager.cc:1428] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:21.778357 11561 ts_tablet_manager.cc:1918] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:21.778518 11561 log.cc:1196] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/a09dd1da783e492bb00a3f65c66d2052
I20260810 18:49:21.778932 11561 ts_tablet_manager.cc:1939] T a09dd1da783e492bb00a3f65c66d2052 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting consensus metadata
I20260810 18:49:21.780181 10666 catalog_manager.cc:4841] TS a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333): tablet a09dd1da783e492bb00a3f65c66d2052 (table test-table [id=ccacd70450034f9297b4dcb0dc44e739]) successfully deleted
I20260810 18:49:21.816250 11534 raft_consensus.cc:482] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:21.816501 11534 raft_consensus.cc:504] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.818320 11534 leader_election.cc:288] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333)
I20260810 18:49:21.824612 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.818510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.29.174.195:37857)
0810 18:49:21.818731 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.818753 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.818881 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.819410 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.819415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.819448 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.819675 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.819686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.820963 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.820968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.822283 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.822301 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.823882 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.823888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.824069 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.824104 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.824194 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.824353 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 18:49:21.824635 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.818682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:37857 (local address 127.29.174.193:44395)
0810 18:49:21.818872 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.818879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.818897 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:49:21.819070 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.819076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.819152 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.819229 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.819831 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.820846 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.822478 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.823727 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.823891 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.823966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.824228 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.824366 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:49:21.825428 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f847cc9d9b2f458c9a461707d3dada67" candidate_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" is_pre_election: true
I20260810 18:49:21.825685 10940 raft_consensus.cc:2454] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1ebabf309e4c5283ae2fbafd40625a in term 0.
I20260810 18:49:21.825735 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.819185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56285 (local address 127.29.174.194:33333)
0810 18:49:21.819369 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.819375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.819953 (+ 578us) server_negotiation.cc:407] Connection header received
0810 18:49:21.820109 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.820116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.820189 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.820279 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.820802 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.821906 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.823642 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.824789 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.824941 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.825013 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.825319 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.825447 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:49:21.826304 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.818860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:33333 (local address 127.29.174.195:56285)
0810 18:49:21.819844 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.819861 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.819984 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.820310 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.820315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.820341 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.820653 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.820664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.822053 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.822058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.823486 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.823499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.824998 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.825004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.825142 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.825164 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.825901 (+ 737us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.826032 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"thread_start_us":663,"threads_started":1}
I20260810 18:49:21.826241 11272 leader_election.cc:302] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, cd1ebabf309e4c5283ae2fbafd40625a; no voters:
I20260810 18:49:21.826682 11534 raft_consensus.cc:2781] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:21.826795 11534 raft_consensus.cc:482] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:21.826877 11534 raft_consensus.cc:3037] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.827426 11161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f847cc9d9b2f458c9a461707d3dada67" candidate_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" is_pre_election: true
I20260810 18:49:21.827880 11161 raft_consensus.cc:2454] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1ebabf309e4c5283ae2fbafd40625a in term 0.
I20260810 18:49:21.829348 11534 raft_consensus.cc:504] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.829931 11534 leader_election.cc:288] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [CANDIDATE]: Term 1 election: Requested vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333)
I20260810 18:49:21.830387 11161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f847cc9d9b2f458c9a461707d3dada67" candidate_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e60ef0d8874e8e80b8775451cabfc2"
I20260810 18:49:21.830470 10940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f847cc9d9b2f458c9a461707d3dada67" candidate_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
I20260810 18:49:21.830586 11161 raft_consensus.cc:3037] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.830659 10940 raft_consensus.cc:3037] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.832536 10940 raft_consensus.cc:2454] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1ebabf309e4c5283ae2fbafd40625a in term 1.
I20260810 18:49:21.832861 11272 leader_election.cc:302] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, cd1ebabf309e4c5283ae2fbafd40625a; no voters:
I20260810 18:49:21.833104 11534 raft_consensus.cc:2781] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:21.833519 11534 raft_consensus.cc:686] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 LEADER]: Becoming Leader. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Running, Role: LEADER
I20260810 18:49:21.833875 11534 consensus_queue.cc:237] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.833995 11161 raft_consensus.cc:2454] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1ebabf309e4c5283ae2fbafd40625a in term 1.
I20260810 18:49:21.837177 10692 catalog_manager.cc:5495] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195). New cstate: current_term: 1 leader_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } health_report { overall_health: HEALTHY } } }
Aug 10 18:49:21 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387759, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.254@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:21.890472 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.876468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.29.174.254:43437)
0810 18:49:21.876700 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.876706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.877172 (+ 466us) server_negotiation.cc:407] Connection header received
0810 18:49:21.877339 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.877343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.877398 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.877478 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:21.878900 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.879638 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.881449 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.881819 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.881901 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.885387 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:21.885392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:21.885395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:21.885421 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:21.887918 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.888577 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.888582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.888584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.888687 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:21.889073 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:21.889077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:21.889081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:21.889325 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:21.889491 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.889881 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.890130 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":102}
I20260810 18:49:21.890502 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.876472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43437 (local address 127.0.0.1:59488)
0810 18:49:21.877073 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.877092 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.877225 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.877518 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.877523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.878432 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:21.878770 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.878783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.879764 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.879769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.881323 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.881338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.881976 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.881982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.882149 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.882811 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:21.882835 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:21.885217 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:21.888021 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.888033 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.888040 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.888471 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.888786 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:21.888791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:21.888795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:21.888980 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:21.889453 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:21.889464 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:21.889620 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.889757 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.890016 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":307,"threads_started":1}
I20260810 18:49:21.894912 10692 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59488:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:49:21.895874 10692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:21.901991 10901 tablet_service.cc:1467] Processing CreateTablet for tablet dd6116ef184f4fef930444d6efe7fab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=24295e7d2a6a4d23b48736c0a7e2715c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:49:21.902034 11124 tablet_service.cc:1467] Processing CreateTablet for tablet dd6116ef184f4fef930444d6efe7fab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=24295e7d2a6a4d23b48736c0a7e2715c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:49:21.902484 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6116ef184f4fef930444d6efe7fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.902611 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6116ef184f4fef930444d6efe7fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.907677 11525 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: Bootstrap starting.
I20260810 18:49:21.908154 11364 tablet_service.cc:1467] Processing CreateTablet for tablet dd6116ef184f4fef930444d6efe7fab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=24295e7d2a6a4d23b48736c0a7e2715c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:49:21.908672 11364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd6116ef184f4fef930444d6efe7fab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:21.909273 11525 tablet_bootstrap.cc:654] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.909969 11523 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: Bootstrap starting.
I20260810 18:49:21.911646 11523 tablet_bootstrap.cc:654] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.915158 11523 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: No bootstrap required, opened a new log
I20260810 18:49:21.915324 11523 ts_tablet_manager.cc:1397] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:49:21.916092 11523 raft_consensus.cc:348] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.916333 11523 raft_consensus.cc:374] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.916424 11523 raft_consensus.cc:729] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.916657 11523 consensus_queue.cc:260] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.917130 11529 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: Bootstrap starting.
I20260810 18:49:21.917979 11542 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:21.918308 11542 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:21.918778 11529 tablet_bootstrap.cc:654] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:21.918771 11523 ts_tablet_manager.cc:1428] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:49:21.920972 11525 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: No bootstrap required, opened a new log
I20260810 18:49:21.921144 11525 ts_tablet_manager.cc:1397] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260810 18:49:21.921808 11525 raft_consensus.cc:348] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.922024 11525 raft_consensus.cc:374] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.922088 11525 raft_consensus.cc:729] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.922310 11525 consensus_queue.cc:260] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.922914 11525 ts_tablet_manager.cc:1428] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:21.922785 11538 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:21.923172 11538 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:21.924891 11529 tablet_bootstrap.cc:492] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: No bootstrap required, opened a new log
I20260810 18:49:21.925051 11529 ts_tablet_manager.cc:1397] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260810 18:49:21.925686 11529 raft_consensus.cc:348] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.925877 11529 raft_consensus.cc:374] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:21.925956 11529 raft_consensus.cc:729] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Initialized, Role: FOLLOWER
I20260810 18:49:21.926165 11529 consensus_queue.cc:260] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.926558 11534 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:21.926828 11534 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:21.927276 11529 ts_tablet_manager.cc:1428] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:49:21.964629 11538 raft_consensus.cc:482] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:21.964841 11538 raft_consensus.cc:504] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.966463 11538 leader_election.cc:288] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
W20260810 18:49:21.970569 11014 tablet.cc:2369] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:21.974112 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.966763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:51725 (local address 127.29.174.194:33333)
0810 18:49:21.966981 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.966987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.967866 (+ 879us) server_negotiation.cc:407] Connection header received
0810 18:49:21.968456 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.968462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.968534 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.968622 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.969352 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.970114 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.972137 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.973086 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.973271 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.973350 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.973702 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.973846 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:49:21.974267 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.966619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:33333 (local address 127.29.174.193:51725)
0810 18:49:21.967767 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.967791 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.968030 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.968662 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.968666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.968690 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.969154 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.969172 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.970243 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.970248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.971982 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.971994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.973341 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.973348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.973539 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.973575 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.973670 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.973795 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"mutex_wait_us":47,"thread_start_us":1391,"threads_started":1}
I20260810 18:49:21.975059 11161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6116ef184f4fef930444d6efe7fab6" candidate_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" is_pre_election: true
I20260810 18:49:21.975350 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.967540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:33927 (local address 127.29.174.195:42183)
0810 18:49:21.967696 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:21.967702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:21.968388 (+ 686us) server_negotiation.cc:407] Connection header received
0810 18:49:21.968648 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:21.968653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:21.968717 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:21.968798 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:21.969454 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.970579 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.973069 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.974160 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.974298 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.974387 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:21.974978 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 18:49:21.975103 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:49:21.975322 11161 raft_consensus.cc:2454] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2dcc03c1864e19a055dd2eb60116d2 in term 0.
I20260810 18:49:21.975540 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:21.967046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:42183 (local address 127.29.174.193:33927)
0810 18:49:21.968282 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:21.968310 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:21.968509 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:21.968825 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:21.968830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:21.968860 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:21.969280 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.969294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.970740 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.970745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.972914 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:21.972929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:21.974659 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:21.974665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:21.974831 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:21.974865 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:49:21.974950 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:49:21.975066 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":32,"thread_start_us":647,"threads_started":1}
I20260810 18:49:21.975870 10824 leader_election.cc:302] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2; no voters:
I20260810 18:49:21.976230 11538 raft_consensus.cc:2781] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:21.976209 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6116ef184f4fef930444d6efe7fab6" candidate_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" is_pre_election: true
I20260810 18:49:21.976348 11538 raft_consensus.cc:482] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:21.976429 11538 raft_consensus.cc:3037] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.976459 11398 raft_consensus.cc:2454] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2dcc03c1864e19a055dd2eb60116d2 in term 0.
W20260810 18:49:21.978300 11480 tablet.cc:2369] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:21.978623 11538 raft_consensus.cc:504] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:21.979137 11538 leader_election.cc:288] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [CANDIDATE]: Term 1 election: Requested vote from peers a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:21.979497 11161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6116ef184f4fef930444d6efe7fab6" candidate_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4e60ef0d8874e8e80b8775451cabfc2"
I20260810 18:49:21.979579 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd6116ef184f4fef930444d6efe7fab6" candidate_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
I20260810 18:49:21.979714 11161 raft_consensus.cc:3037] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.979749 11398 raft_consensus.cc:3037] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:21.981993 11161 raft_consensus.cc:2454] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2dcc03c1864e19a055dd2eb60116d2 in term 1.
I20260810 18:49:21.982332 10824 leader_election.cc:302] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2; no voters:
I20260810 18:49:21.982581 11398 raft_consensus.cc:2454] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2dcc03c1864e19a055dd2eb60116d2 in term 1.
I20260810 18:49:21.982604 11538 raft_consensus.cc:2781] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:21.986649 11538 raft_consensus.cc:686] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Running, Role: LEADER
I20260810 18:49:21.987083 11538 consensus_queue.cc:237] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
W20260810 18:49:21.988233 11245 tablet.cc:2369] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:49:21.996191 11604 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: TxnStatusTablet state changed. Reason: New leader 1a2dcc03c1864e19a055dd2eb60116d2. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:21.996496 11604 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:49:21.996783 10694 catalog_manager.cc:5495] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193). New cstate: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:22.004832 11606 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:49:22.005075 11606 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:49:22.033545 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.025999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.0.0.1:59344)
0810 18:49:22.026199 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:22.026218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:22.026413 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:22.026969 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:22.026974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:22.026987 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:49:22.027187 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.027199 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.028651 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.028657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.030192 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.030208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.031190 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.031196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.031436 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.031516 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.032944 (+ 1428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.032968 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:49:22.033060 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:49:22.033218 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 18:49:22.033540 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.025886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.29.174.193:44395)
0810 18:49:22.026072 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.026078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.026299 (+ 221us) server_negotiation.cc:407] Connection header received
0810 18:49:22.026630 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.026636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.026765 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.026909 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:22.027343 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.028493 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.030372 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.031022 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.031154 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.032024 (+ 870us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.032813 (+ 789us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.032963 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.033093 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.033179 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:49:22.037928 11398 raft_consensus.cc:1261] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Refusing update from remote peer 1a2dcc03c1864e19a055dd2eb60116d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.038231 11161 raft_consensus.cc:1261] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Refusing update from remote peer 1a2dcc03c1864e19a055dd2eb60116d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.038805 11538 consensus_queue.cc:1035] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.039448 11538 consensus_queue.cc:1035] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.049180 11534 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: TxnStatusTablet state changed. Reason: New leader 1a2dcc03c1864e19a055dd2eb60116d2. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.049463 11534 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:22.049945 11542 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: TxnStatusTablet state changed. Reason: New leader 1a2dcc03c1864e19a055dd2eb60116d2. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.050424 11542 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:22.054474 11538 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.054759 11538 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:49:22.054983 11538 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.055047 11542 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.055190 11538 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P 1a2dcc03c1864e19a055dd2eb60116d2: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:49:22.055281 11542 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P a4e60ef0d8874e8e80b8775451cabfc2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:22.059773 11534 tablet_replica.cc:439] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } } }
I20260810 18:49:22.060048 11534 tablet_replica.cc:442] T dd6116ef184f4fef930444d6efe7fab6 P cd1ebabf309e4c5283ae2fbafd40625a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:49:22.067497 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.062314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.29.174.195:42183)
0810 18:49:22.062491 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.062496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.062736 (+ 240us) server_negotiation.cc:407] Connection header received
0810 18:49:22.062938 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.062943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.063013 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.063130 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:22.063490 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.064168 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.065432 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.065864 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.065969 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.066422 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.066910 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.067041 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.067129 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.067210 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:49:22.087234 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.076179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:54311 (local address 127.29.174.193:44395)
0810 18:49:22.076319 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.076322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.082881 (+ 6559us) server_negotiation.cc:407] Connection header received
0810 18:49:22.083051 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.083055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.083115 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.083185 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:22.083608 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.084282 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.085543 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.086023 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.086129 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.086545 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.086635 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.086762 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.086869 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.086940 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 18:49:22.087515 11619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.076080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.29.174.195:54311)
0810 18:49:22.082779 (+ 6699us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:22.082801 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:22.082930 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:22.083214 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:22.083218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:22.083230 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:49:22.083471 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.083479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.084410 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.084415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.085412 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.085423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.086189 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.086196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.086312 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.086389 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.086723 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.086733 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:49:22.086816 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:49:22.086943 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6556,"thread_start_us":6442,"threads_started":1}
I20260810 18:49:22.098420 11161 raft_consensus.cc:1261] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Refusing update from remote peer cd1ebabf309e4c5283ae2fbafd40625a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.098601 10940 raft_consensus.cc:1261] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Refusing update from remote peer cd1ebabf309e4c5283ae2fbafd40625a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.099202 11534 consensus_queue.cc:1035] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.099700 11575 consensus_queue.cc:1035] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.156632 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.150127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:58403 (local address 127.29.174.195:42183)
0810 18:49:22.150331 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.150357 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.150687 (+ 330us) server_negotiation.cc:407] Connection header received
0810 18:49:22.150981 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.150986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.151072 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.151145 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:22.151983 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.152859 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.154426 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.155124 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.155254 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.155894 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.156020 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.156083 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.156342 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.156423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:49:22.157246 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.149648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:42183 (local address 127.29.174.193:58403)
0810 18:49:22.150587 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:22.150608 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:22.150822 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:22.151342 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:22.151347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:22.151359 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:49:22.151775 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.151791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.152998 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.153001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.154262 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:22.154275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.155422 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.155429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:22.155618 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.155687 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.156174 (+ 487us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.156184 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:49:22.156801 (+ 617us) client_negotiation.cc:240] Negotiation successful
0810 18:49:22.156932 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":2326,"threads_started":1}
I20260810 18:49:22.203068 10694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:22.203313 10694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:22.206303 10694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Sending DeleteTablet for 3 replicas of tablet f847cc9d9b2f458c9a461707d3dada67
I20260810 18:49:22.207417 11124 tablet_service.cc:1514] Processing DeleteTablet for tablet f847cc9d9b2f458c9a461707d3dada67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38106
I20260810 18:49:22.207687 11561 tablet_replica.cc:330] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: stopping tablet replica
I20260810 18:49:22.207880 11561 raft_consensus.cc:2229] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:22.208057 11561 raft_consensus.cc:2258] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:22.208019 11364 tablet_service.cc:1514] Processing DeleteTablet for tablet f847cc9d9b2f458c9a461707d3dada67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36338
I20260810 18:49:22.208292 11558 tablet_replica.cc:330] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: stopping tablet replica
I20260810 18:49:22.208489 11558 raft_consensus.cc:2229] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:22.208920 11558 raft_consensus.cc:2258] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:22.209381 11561 ts_tablet_manager.cc:1905] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.209973 11558 ts_tablet_manager.cc:1905] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.212535 10901 tablet_service.cc:1514] Processing DeleteTablet for tablet f847cc9d9b2f458c9a461707d3dada67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59334
I20260810 18:49:22.212810 11560 tablet_replica.cc:330] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: stopping tablet replica
I20260810 18:49:22.213004 11560 raft_consensus.cc:2229] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:22.213177 11560 raft_consensus.cc:2258] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:22.214443 11560 ts_tablet_manager.cc:1905] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.216789 11561 ts_tablet_manager.cc:1918] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:22.216949 11561 log.cc:1196] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/f847cc9d9b2f458c9a461707d3dada67
I20260810 18:49:22.217370 11561 ts_tablet_manager.cc:1939] T f847cc9d9b2f458c9a461707d3dada67 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting consensus metadata
I20260810 18:49:22.218415 10666 catalog_manager.cc:4841] TS a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333): tablet f847cc9d9b2f458c9a461707d3dada67 (table test-table [id=4bf1fb84fa06435bb2beef8363fd6636]) successfully deleted
I20260810 18:49:22.224992 11560 ts_tablet_manager.cc:1918] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:22.225144 11560 log.cc:1196] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/f847cc9d9b2f458c9a461707d3dada67
I20260810 18:49:22.225523 11560 ts_tablet_manager.cc:1939] T f847cc9d9b2f458c9a461707d3dada67 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting consensus metadata
I20260810 18:49:22.226492 10666 catalog_manager.cc:4841] TS 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395): tablet f847cc9d9b2f458c9a461707d3dada67 (table test-table [id=4bf1fb84fa06435bb2beef8363fd6636]) successfully deleted
I20260810 18:49:22.227111 11558 ts_tablet_manager.cc:1918] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:22.227274 11558 log.cc:1196] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/f847cc9d9b2f458c9a461707d3dada67
I20260810 18:49:22.229226 11558 ts_tablet_manager.cc:1939] T f847cc9d9b2f458c9a461707d3dada67 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting consensus metadata
I20260810 18:49:22.230175 10668 catalog_manager.cc:4841] TS cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183): tablet f847cc9d9b2f458c9a461707d3dada67 (table test-table [id=4bf1fb84fa06435bb2beef8363fd6636]) successfully deleted
Aug 10 18:49:22 dist-test-slave-6zbq krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:49:22 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:22 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:49:22.296602 10836 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:22.284090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.29.174.254:43437)
0810 18:49:22.284277 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.284284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.284827 (+ 543us) server_negotiation.cc:407] Connection header received
0810 18:49:22.284998 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.285004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.285091 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.285201 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:22.286836 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.287975 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.289940 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.290620 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.290736 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.296335 (+ 5599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59494: BlockingRecv error: recv got EOF from 127.0.0.1:59494 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 18:49:22 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM
I20260810 18:49:22.317751 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.300431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.29.174.254:43437)
0810 18:49:22.300592 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.300598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.301085 (+ 487us) server_negotiation.cc:407] Connection header received
0810 18:49:22.301243 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.301248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.301328 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.301428 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:22.302855 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.303933 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.305919 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.306615 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.306740 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.311448 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:22.311455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:22.311460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:22.311501 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:22.314873 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:22.315615 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:22.315621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:22.315624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:22.315784 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:22.316196 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:22.316201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:22.316204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:22.316550 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:22.316802 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.317112 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.317340 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:49:22.321540 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:22.322239 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:22.329257 10901 tablet_service.cc:1467] Processing CreateTablet for tablet 002e463ed4004cad94458a080b927c3c (DEFAULT_TABLE table=test-table [id=2776410900014cc5942600a1a720b06d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.329687 11124 tablet_service.cc:1467] Processing CreateTablet for tablet 002e463ed4004cad94458a080b927c3c (DEFAULT_TABLE table=test-table [id=2776410900014cc5942600a1a720b06d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.329846 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002e463ed4004cad94458a080b927c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.330096 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002e463ed4004cad94458a080b927c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.335259 11364 tablet_service.cc:1467] Processing CreateTablet for tablet 002e463ed4004cad94458a080b927c3c (DEFAULT_TABLE table=test-table [id=2776410900014cc5942600a1a720b06d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.335685 11364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002e463ed4004cad94458a080b927c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.338567 11525 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Bootstrap starting.
I20260810 18:49:22.340276 11525 tablet_bootstrap.cc:654] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.346108 11529 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Bootstrap starting.
I20260810 18:49:22.347815 11529 tablet_bootstrap.cc:654] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.350050 11523 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Bootstrap starting.
I20260810 18:49:22.351616 11523 tablet_bootstrap.cc:654] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.361804 11525 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: No bootstrap required, opened a new log
I20260810 18:49:22.362056 11525 ts_tablet_manager.cc:1397] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260810 18:49:22.362294 11529 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: No bootstrap required, opened a new log
I20260810 18:49:22.362632 11529 ts_tablet_manager.cc:1397] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.012s
I20260810 18:49:22.362977 11525 raft_consensus.cc:348] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.363346 11525 raft_consensus.cc:374] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.363492 11525 raft_consensus.cc:729] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.363792 11525 consensus_queue.cc:260] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.363603 11529 raft_consensus.cc:348] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.364102 11529 raft_consensus.cc:374] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.364233 11529 raft_consensus.cc:729] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.364471 11529 consensus_queue.cc:260] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.365083 11529 ts_tablet_manager.cc:1428] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:22.366276 11525 ts_tablet_manager.cc:1428] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 18:49:22.369637 11523 tablet_bootstrap.cc:492] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: No bootstrap required, opened a new log
I20260810 18:49:22.369853 11523 ts_tablet_manager.cc:1397] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.003s
I20260810 18:49:22.370676 11523 raft_consensus.cc:348] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.371021 11523 raft_consensus.cc:374] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.371171 11523 raft_consensus.cc:729] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.371459 11523 consensus_queue.cc:260] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.372807 11523 ts_tablet_manager.cc:1428] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:49:22.459353 11542 raft_consensus.cc:482] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:22.459583 11542 raft_consensus.cc:504] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.460325 11542 leader_election.cc:288] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:22.460945 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002e463ed4004cad94458a080b927c3c" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" is_pre_election: true
I20260810 18:49:22.461009 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002e463ed4004cad94458a080b927c3c" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" is_pre_election: true
I20260810 18:49:22.461278 10939 raft_consensus.cc:2454] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:22.461313 11398 raft_consensus.cc:2454] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:22.461853 11040 leader_election.cc:302] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2; no voters:
I20260810 18:49:22.462318 11542 raft_consensus.cc:2781] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:22.462437 11542 raft_consensus.cc:482] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:22.462533 11542 raft_consensus.cc:3037] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.465399 11542 raft_consensus.cc:504] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.466322 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002e463ed4004cad94458a080b927c3c" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
I20260810 18:49:22.466527 10939 raft_consensus.cc:3037] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.469434 10939 raft_consensus.cc:2454] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:22.469904 11040 leader_election.cc:302] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2; no voters:
I20260810 18:49:22.470497 11542 leader_election.cc:288] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:22.470645 11542 raft_consensus.cc:2781] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:22.470757 11542 raft_consensus.cc:686] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Becoming Leader. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Running, Role: LEADER
I20260810 18:49:22.470960 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002e463ed4004cad94458a080b927c3c" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
I20260810 18:49:22.470983 11542 consensus_queue.cc:237] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.471180 11398 raft_consensus.cc:3037] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.474640 11398 raft_consensus.cc:2454] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:22.491835 10692 catalog_manager.cc:5495] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:22.534281 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:22.524110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.29.174.194:33333)
0810 18:49:22.528255 (+ 4145us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:22.528261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:22.528279 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:49:22.528345 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:22.528350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:22.528428 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:22.528528 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:22.529004 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.530077 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.532094 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:22.532768 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:22.532887 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:22.533421 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:22.533560 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:22.533645 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:22.533816 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 18:49:22.533896 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4008,"thread_start_us":3878,"threads_started":1}
I20260810 18:49:22.537009 11398 raft_consensus.cc:1261] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.537617 10939 raft_consensus.cc:1261] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:22.538300 11664 consensus_queue.cc:1035] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.538707 11542 consensus_queue.cc:1035] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:22.548120 11545 mvcc.cc:205] Tried to move back new op lower bound from 7317044275346112512 to 7317044275087753216. Current Snapshot: MvccSnapshot[applied={T|T < 7317044275346112512}]
I20260810 18:49:22.557128 10694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:22.557343 10694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:22.560184 10694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Sending DeleteTablet for 3 replicas of tablet 002e463ed4004cad94458a080b927c3c
I20260810 18:49:22.560917 10901 tablet_service.cc:1514] Processing DeleteTablet for tablet 002e463ed4004cad94458a080b927c3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59334
I20260810 18:49:22.561331 11560 tablet_replica.cc:330] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: stopping tablet replica
I20260810 18:49:22.561304 11364 tablet_service.cc:1514] Processing DeleteTablet for tablet 002e463ed4004cad94458a080b927c3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36338
I20260810 18:49:22.561510 11560 raft_consensus.cc:2229] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:22.561578 11558 tablet_replica.cc:330] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: stopping tablet replica
I20260810 18:49:22.561695 11560 raft_consensus.cc:2258] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:22.561743 11558 raft_consensus.cc:2229] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:22.561956 11558 raft_consensus.cc:2258] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:22.562288 11124 tablet_service.cc:1514] Processing DeleteTablet for tablet 002e463ed4004cad94458a080b927c3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:22 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38106
I20260810 18:49:22.562589 11561 tablet_replica.cc:330] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: stopping tablet replica
I20260810 18:49:22.562764 11561 raft_consensus.cc:2229] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:22.562561 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:49:22.563077 11560 ts_tablet_manager.cc:1905] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.563063 11561 raft_consensus.cc:2258] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 18:49:22.563195 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:22.563344 11558 ts_tablet_manager.cc:1905] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.567478 11561 ts_tablet_manager.cc:1905] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:22.569183 11560 ts_tablet_manager.cc:1918] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:22.569303 11560 log.cc:1196] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/002e463ed4004cad94458a080b927c3c
I20260810 18:49:22.569689 11560 ts_tablet_manager.cc:1939] T 002e463ed4004cad94458a080b927c3c P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting consensus metadata
I20260810 18:49:22.570727 10901 tablet_service.cc:1467] Processing CreateTablet for tablet 26ccfe31700d441082b868c60d56e5e7 (DEFAULT_TABLE table=test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.570825 10666 catalog_manager.cc:4841] TS 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395): tablet 002e463ed4004cad94458a080b927c3c (table test-table [id=2776410900014cc5942600a1a720b06d]) successfully deleted
I20260810 18:49:22.571074 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ccfe31700d441082b868c60d56e5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.571522 11364 tablet_service.cc:1467] Processing CreateTablet for tablet 26ccfe31700d441082b868c60d56e5e7 (DEFAULT_TABLE table=test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.571959 11364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ccfe31700d441082b868c60d56e5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.576673 11561 ts_tablet_manager.cc:1918] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:22.576808 11561 log.cc:1196] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/002e463ed4004cad94458a080b927c3c
I20260810 18:49:22.577169 11561 ts_tablet_manager.cc:1939] T 002e463ed4004cad94458a080b927c3c P a4e60ef0d8874e8e80b8775451cabfc2: Deleting consensus metadata
I20260810 18:49:22.577904 11525 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Bootstrap starting.
I20260810 18:49:22.579416 11525 tablet_bootstrap.cc:654] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.579658 11124 tablet_service.cc:1467] Processing CreateTablet for tablet 26ccfe31700d441082b868c60d56e5e7 (DEFAULT_TABLE table=test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:22.580053 11124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ccfe31700d441082b868c60d56e5e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:22.581219 11558 ts_tablet_manager.cc:1918] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:22.581427 11558 log.cc:1196] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/002e463ed4004cad94458a080b927c3c
I20260810 18:49:22.581928 11558 ts_tablet_manager.cc:1939] T 002e463ed4004cad94458a080b927c3c P cd1ebabf309e4c5283ae2fbafd40625a: Deleting consensus metadata
I20260810 18:49:22.585125 10668 catalog_manager.cc:4841] TS cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183): tablet 002e463ed4004cad94458a080b927c3c (table test-table [id=2776410900014cc5942600a1a720b06d]) successfully deleted
I20260810 18:49:22.585258 10666 catalog_manager.cc:4841] TS a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333): tablet 002e463ed4004cad94458a080b927c3c (table test-table [id=2776410900014cc5942600a1a720b06d]) successfully deleted
I20260810 18:49:22.586265 11523 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Bootstrap starting.
I20260810 18:49:22.587033 11529 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Bootstrap starting.
I20260810 18:49:22.587869 11523 tablet_bootstrap.cc:654] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.588657 11529 tablet_bootstrap.cc:654] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:22.588636 11525 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: No bootstrap required, opened a new log
I20260810 18:49:22.588933 11525 ts_tablet_manager.cc:1397] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.001s
I20260810 18:49:22.589675 11525 raft_consensus.cc:348] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.589977 11525 raft_consensus.cc:374] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.590107 11525 raft_consensus.cc:729] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2dcc03c1864e19a055dd2eb60116d2, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.590436 11525 consensus_queue.cc:260] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.590996 11525 ts_tablet_manager.cc:1428] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:22.596982 11529 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: No bootstrap required, opened a new log
I20260810 18:49:22.596987 11523 tablet_bootstrap.cc:492] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: No bootstrap required, opened a new log
I20260810 18:49:22.597175 11523 ts_tablet_manager.cc:1397] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 18:49:22.597214 11529 ts_tablet_manager.cc:1397] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260810 18:49:22.597949 11529 raft_consensus.cc:348] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.597941 11523 raft_consensus.cc:348] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.598193 11529 raft_consensus.cc:374] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.598224 11523 raft_consensus.cc:374] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:22.598291 11523 raft_consensus.cc:729] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.598297 11529 raft_consensus.cc:729] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1ebabf309e4c5283ae2fbafd40625a, State: Initialized, Role: FOLLOWER
I20260810 18:49:22.598547 11523 consensus_queue.cc:260] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.598547 11529 consensus_queue.cc:260] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.599061 11529 ts_tablet_manager.cc:1428] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:49:22.599581 11523 ts_tablet_manager.cc:1428] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:49:22.941066 11672 raft_consensus.cc:482] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:22.941336 11672 raft_consensus.cc:504] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.942058 11672 leader_election.cc:288] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:22.942663 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ccfe31700d441082b868c60d56e5e7" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" is_pre_election: true
I20260810 18:49:22.942667 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ccfe31700d441082b868c60d56e5e7" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" is_pre_election: true
I20260810 18:49:22.942934 10939 raft_consensus.cc:2454] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:22.942934 11398 raft_consensus.cc:2454] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 0.
I20260810 18:49:22.943440 11042 leader_election.cc:302] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2, cd1ebabf309e4c5283ae2fbafd40625a; no voters:
I20260810 18:49:22.943725 11672 raft_consensus.cc:2781] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:22.943848 11672 raft_consensus.cc:482] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:22.943931 11672 raft_consensus.cc:3037] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.945981 11672 raft_consensus.cc:504] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.946542 11672 leader_election.cc:288] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Requested vote from peers 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395), cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183)
I20260810 18:49:22.946941 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ccfe31700d441082b868c60d56e5e7" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a2dcc03c1864e19a055dd2eb60116d2"
I20260810 18:49:22.947108 10939 raft_consensus.cc:3037] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.947038 11398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ccfe31700d441082b868c60d56e5e7" candidate_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1ebabf309e4c5283ae2fbafd40625a"
I20260810 18:49:22.947256 11398 raft_consensus.cc:3037] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:22.949369 10939 raft_consensus.cc:2454] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:22.949553 11398 raft_consensus.cc:2454] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4e60ef0d8874e8e80b8775451cabfc2 in term 1.
I20260810 18:49:22.949716 11040 leader_election.cc:302] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2dcc03c1864e19a055dd2eb60116d2, a4e60ef0d8874e8e80b8775451cabfc2; no voters:
I20260810 18:49:22.949986 11672 raft_consensus.cc:2781] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:22.950140 11672 raft_consensus.cc:686] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Becoming Leader. State: Replica: a4e60ef0d8874e8e80b8775451cabfc2, State: Running, Role: LEADER
I20260810 18:49:22.950426 11672 consensus_queue.cc:237] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } }
I20260810 18:49:22.952709 10694 catalog_manager.cc:5495] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4e60ef0d8874e8e80b8775451cabfc2" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 33333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:23.060165 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.055787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:40831 (local address 127.29.174.193:44395)
0810 18:49:23.056178 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:23.056182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:23.056202 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:49:23.056356 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:23.056359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:23.056421 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:23.056503 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:23.056837 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.057519 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.058791 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.059205 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.059309 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.059610 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:23.059705 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:23.059776 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:23.059861 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 18:49:23.059931 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":193,"threads_started":1}
I20260810 18:49:23.060194 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.055661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44395 (local address 127.29.174.194:40831)
0810 18:49:23.056134 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:23.056150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:23.056257 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:23.056516 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:23.056519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:23.056527 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:49:23.056732 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:23.056740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.057614 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.057617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:23.058668 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:23.058681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.059314 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.059318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:23.059429 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.059483 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:49:23.059781 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:49:23.059788 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:49:23.059838 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 18:49:23.059930 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
I20260810 18:49:23.067035 10939 raft_consensus.cc:1261] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:23.067068 11398 raft_consensus.cc:1261] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Refusing update from remote peer a4e60ef0d8874e8e80b8775451cabfc2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:23.067561 11672 consensus_queue.cc:1035] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1ebabf309e4c5283ae2fbafd40625a" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:23.067827 11542 consensus_queue.cc:1035] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a2dcc03c1864e19a055dd2eb60116d2" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:23.090821 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.085421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:42663 (local address 127.29.174.194:33333)
0810 18:49:23.086123 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:23.086128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:23.086147 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:49:23.086202 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:23.086206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:23.086280 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:23.086407 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:23.086744 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.087520 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.089005 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.089555 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.089666 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.090066 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:23.090161 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:23.090242 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:23.090396 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 18:49:23.090473 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":466,"threads_started":1}
I20260810 18:49:23.090843 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.085350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:33333 (local address 127.29.174.193:42663)
0810 18:49:23.085877 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:23.085897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:23.086019 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:23.086423 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:23.086428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:23.086439 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:49:23.086625 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:23.086633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.087618 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.087621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:23.088881 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:23.088894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.089701 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.089708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:23.089863 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.089899 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:49:23.090253 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:49:23.090264 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:49:23.090335 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:49:23.090485 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260810 18:49:23.117620 10692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:23.117856 10692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:23.120447 10692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4c3defda7ac4df99a2e4b35b13c36de: Sending DeleteTablet for 3 replicas of tablet 26ccfe31700d441082b868c60d56e5e7
I20260810 18:49:23.121141 10901 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ccfe31700d441082b868c60d56e5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:23 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59334
I20260810 18:49:23.121141 11124 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ccfe31700d441082b868c60d56e5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:23 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38106
I20260810 18:49:23.121241 11364 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ccfe31700d441082b868c60d56e5e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:23 UTC) from {username='oryx', principal='oryx/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:36338
I20260810 18:49:23.121789 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 10800
I20260810 18:49:23.133469 11707 tablet_replica.cc:330] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: stopping tablet replica
I20260810 18:49:23.133675 11705 tablet_replica.cc:330] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: stopping tablet replica
I20260810 18:49:23.133821 11707 raft_consensus.cc:2229] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:23.133880 11704 tablet_replica.cc:330] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: stopping tablet replica
I20260810 18:49:23.133860 11705 raft_consensus.cc:2229] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:23.134074 11704 raft_consensus.cc:2229] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:23.134110 11705 raft_consensus.cc:2258] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:23.134130 11707 raft_consensus.cc:2258] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:23.134855 11704 raft_consensus.cc:2258] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:23.135134 11705 ts_tablet_manager.cc:1905] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:23.135185 11707 ts_tablet_manager.cc:1905] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:23.135900 11704 ts_tablet_manager.cc:1905] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:23.142755 11705 ts_tablet_manager.cc:1918] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:23.142753 11707 ts_tablet_manager.cc:1918] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:23.142892 11705 log.cc:1196] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/26ccfe31700d441082b868c60d56e5e7
I20260810 18:49:23.143013 11707 log.cc:1196] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/26ccfe31700d441082b868c60d56e5e7
I20260810 18:49:23.143326 11705 ts_tablet_manager.cc:1939] T 26ccfe31700d441082b868c60d56e5e7 P 1a2dcc03c1864e19a055dd2eb60116d2: Deleting consensus metadata
I20260810 18:49:23.143437 11704 ts_tablet_manager.cc:1918] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:49:23.143565 11704 log.cc:1196] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/26ccfe31700d441082b868c60d56e5e7
I20260810 18:49:23.143563 11707 ts_tablet_manager.cc:1939] T 26ccfe31700d441082b868c60d56e5e7 P cd1ebabf309e4c5283ae2fbafd40625a: Deleting consensus metadata
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.193@KRBTEST.COM
I20260810 18:49:23.144028 11704 ts_tablet_manager.cc:1939] T 26ccfe31700d441082b868c60d56e5e7 P a4e60ef0d8874e8e80b8775451cabfc2: Deleting consensus metadata
I20260810 18:49:23.144459 10666 catalog_manager.cc:4841] TS 1a2dcc03c1864e19a055dd2eb60116d2 (127.29.174.193:44395): tablet 26ccfe31700d441082b868c60d56e5e7 (table test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]) successfully deleted
I20260810 18:49:23.144780 10668 catalog_manager.cc:4841] TS cd1ebabf309e4c5283ae2fbafd40625a (127.29.174.195:42183): tablet 26ccfe31700d441082b868c60d56e5e7 (table test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]) successfully deleted
I20260810 18:49:23.145031 10666 catalog_manager.cc:4841] TS a4e60ef0d8874e8e80b8775451cabfc2 (127.29.174.194:33333): tablet 26ccfe31700d441082b868c60d56e5e7 (table test-table [id=b6f0f2f3be12489caaa486ee4c04dd2b]) successfully deleted
I20260810 18:49:23.149957 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.123048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.29.174.193:44395)
0810 18:49:23.123221 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:23.123227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:23.132801 (+ 9574us) server_negotiation.cc:407] Connection header received
0810 18:49:23.132986 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:23.132992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:23.133086 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:23.133188 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:23.135182 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.136212 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.138979 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.139614 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.139731 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.145068 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:23.145087 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:23.145106 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:23.145143 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:23.147660 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.148222 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.148228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.148230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.148377 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.148705 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.148709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.148711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.149007 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:23.149236 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:23.149476 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:49:23.149663 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:49:23.150581 11006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:23.271561 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 10800
I20260810 18:49:23.300173 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 11018
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.194@KRBTEST.COM
I20260810 18:49:23.313833 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.300871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.29.174.194:33333)
0810 18:49:23.301060 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:23.301064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:23.301095 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:49:23.301235 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:23.301239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:23.301304 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:23.301384 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:23.302567 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.303310 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.304603 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.305000 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.305090 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.309062 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:23.309074 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:23.309090 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:23.309139 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:23.311774 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.312282 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.312288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.312290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.312406 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.312651 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.312653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.312655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.312963 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:23.313183 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:23.313380 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 18:49:23.313532 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:49:23.314483 11234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:23.433171 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 11018
I20260810 18:49:23.459056 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 11249
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.174.195@KRBTEST.COM
I20260810 18:49:23.474259 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:23.459809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.29.174.195:42183)
0810 18:49:23.460322 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:23.460326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:23.460341 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:49:23.460394 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:23.460398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:23.460461 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:23.460558 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:23.461920 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.462748 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.464003 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:23.464427 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:23.464541 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:23.468864 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:23.468880 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:23.468897 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:23.468935 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:23.471609 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.472208 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.472215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.472217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.472345 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:23.472723 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:23.472729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:23.472732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:23.473090 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:23.473385 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:23.473683 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:49:23.473897 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":258,"threads_started":1}
I20260810 18:49:23.475119 11470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:23.606729 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 11249
I20260810 18:49:23.630438 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 10635
I20260810 18:49:23.631505 10749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:23.745757 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 10635
2026-08-10T18:49:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5916 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[11723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[11723](info): set up 2 sockets
Aug 10 18:49:23 dist-test-slave-6zbq krb5kdc[11723](info): commencing operation
krb5kdc: starting...
W20260810 18:49:24.971467 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.012s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:25 dist-test-slave-6zbq krb5kdc[11723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:49:25.191751 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46439
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:46439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:25.504638 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:25.504889 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:25.505000 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:25.512437 11753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:25.512533 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:25.512605 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:25.512684 11753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:25.512751 11753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:25.522297 11753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:46439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:25.523396 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:25.524569 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:25.535147 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:25.535214 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:25.535241 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:25.536240 11753 server_base.cc:1034] running on GCE node
I20260810 18:49:25.536888 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:25.538089 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:25.539312 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786387765539268 us; error 23 us; skew 500 ppm
I20260810 18:49:25.539644 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T18:49:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1858 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:25 dist-test-slave-6zbq krb5kdc[11777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:25 dist-test-slave-6zbq krb5kdc[11777](info): set up 2 sockets
Aug 10 18:49:25 dist-test-slave-6zbq krb5kdc[11777](info): commencing operation
krb5kdc: starting...
W20260810 18:49:26.852535 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:27 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:49:27.083458 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45929
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:27.403851 11798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:27.404091 11798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:27.404218 11798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:27.410962 11798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:27.411049 11798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:27.411103 11798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:27.411182 11798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:27.411239 11798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:27.420325 11798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:27.421446 11798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:27.422689 11798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:27.433431 11805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:27.433461 11812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:27.433485 11809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:27.434113 11798 server_base.cc:1034] running on GCE node
I20260810 18:49:27.434876 11798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:27.436089 11798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:27.437286 11798 hybrid_clock.cc:648] HybridClock initialized: now 1786387767437230 us; error 48 us; skew 500 ppm
I20260810 18:49:27.437590 11798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:27 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:27.445365 11798 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:49:27.446729 11798 webserver.cc:460] Webserver started at http://127.29.174.254:35615/ using document root <none> and password file <none>
I20260810 18:49:27.447317 11798 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:27.447409 11798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:27.447659 11798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:27.450326 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "d099cf0cf73845b1b0a755fb50cf943e"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "ae156949d6a498ad2abdb8d8798de5d7"
server_key_iv: "2506df0b068988d805eaa250daa50aeb"
server_key_version: "encryptionkey@0"
I20260810 18:49:27.451058 11798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "d099cf0cf73845b1b0a755fb50cf943e"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "ae156949d6a498ad2abdb8d8798de5d7"
server_key_iv: "2506df0b068988d805eaa250daa50aeb"
server_key_version: "encryptionkey@0"
I20260810 18:49:27.455806 11798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:27.458823 11823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:27.459954 11798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:49:27.460098 11798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "d099cf0cf73845b1b0a755fb50cf943e"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "ae156949d6a498ad2abdb8d8798de5d7"
server_key_iv: "2506df0b068988d805eaa250daa50aeb"
server_key_version: "encryptionkey@0"
I20260810 18:49:27.460263 11798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:27.481207 11798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:27.484814 11798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:27.485064 11798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:27.501711 11798 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:45929
I20260810 18:49:27.501742 11902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:45929 every 8 connection(s)
I20260810 18:49:27.502971 11798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:27.504241 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 11798
I20260810 18:49:27.504472 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:27.505285 30395 external_mini_cluster.cc:1428] Setting key 843f4363fc8eb287009792f253a7cffd
I20260810 18:49:27.507362 11904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:49:27 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:27.520048 11904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Bootstrap starting.
I20260810 18:49:27.523166 11904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:27.524331 11904 log.cc:824] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:27.527441 11904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: No bootstrap required, opened a new log
I20260810 18:49:27.527658 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:27.507229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.29.174.254:45929)
0810 18:49:27.507945 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:27.507957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:27.507995 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:49:27.509623 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:27.509643 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:27.509992 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:27.510302 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:27.512077 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:27.513365 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:27.515022 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:27.515653 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:27.515769 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:27.520395 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:27.520403 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:27.520418 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:27.520455 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:27.523411 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:27.524172 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:27.524179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:27.524184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:27.524350 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:27.524743 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:27.524746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:27.524748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:27.525282 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:27.525497 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:27.526015 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 18:49:27.526445 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":328,"threads_started":1}
I20260810 18:49:27.534948 11904 raft_consensus.cc:348] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:27.535254 11904 raft_consensus.cc:374] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:27.535322 11904 raft_consensus.cc:729] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d099cf0cf73845b1b0a755fb50cf943e, State: Initialized, Role: FOLLOWER
I20260810 18:49:27.535771 11904 consensus_queue.cc:260] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:27.535957 11904 raft_consensus.cc:388] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:27.536082 11904 raft_consensus.cc:482] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:27.536243 11904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:27.538249 11904 raft_consensus.cc:504] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:27.538743 11904 leader_election.cc:302] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d099cf0cf73845b1b0a755fb50cf943e; no voters:
I20260810 18:49:27.539309 11904 leader_election.cc:288] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:27.539455 11913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:27.540122 11913 raft_consensus.cc:686] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 LEADER]: Becoming Leader. State: Replica: d099cf0cf73845b1b0a755fb50cf943e, State: Running, Role: LEADER
I20260810 18:49:27.540746 11913 consensus_queue.cc:237] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:27.541236 11904 sys_catalog.cc:564] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:27.543248 11918 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d099cf0cf73845b1b0a755fb50cf943e. Latest consensus state: current_term: 1 leader_uuid: "d099cf0cf73845b1b0a755fb50cf943e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } } }
I20260810 18:49:27.543327 11917 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d099cf0cf73845b1b0a755fb50cf943e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } } }
I20260810 18:49:27.543655 11918 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:27.544615 11917 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:27.547401 11923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:27.552107 11923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:27.560802 11923 catalog_manager.cc:1347] Generated new cluster ID: 9dcca827321e447492e10d6e123fbb01
I20260810 18:49:27.560930 11923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:27.573429 11923 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:27.575228 11923 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:27.586645 11923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Generated new TSK 0
I20260810 18:49:27.587581 11923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:49:27.647751 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:27.959363 11949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:27.959610 11949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:27.959733 11949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:27.966814 11949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:27.966908 11949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:27.967056 11949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:27.976241 11949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:27.977362 11949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:27.978569 11949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:27.990747 11964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:27.990790 11961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:27.990813 11960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:27.991463 11949 server_base.cc:1034] running on GCE node
I20260810 18:49:27.991993 11949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:27.992802 11949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:27.993978 11949 hybrid_clock.cc:648] HybridClock initialized: now 1786387767993930 us; error 45 us; skew 500 ppm
I20260810 18:49:27.994256 11949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:27 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:27.997149 11949 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:49:27.998461 11949 webserver.cc:460] Webserver started at http://127.29.174.193:39511/ using document root <none> and password file <none>
I20260810 18:49:27.999068 11949 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:27.999179 11949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:27.999475 11949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:28.002187 11949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "c3a71431cac04f78b11938d2d0372e02"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "568eb87e223e9b17027c4a5b155927e4"
server_key_iv: "e690269015fca5cbbdecd234bc33e8b4"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.002966 11949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "c3a71431cac04f78b11938d2d0372e02"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "568eb87e223e9b17027c4a5b155927e4"
server_key_iv: "e690269015fca5cbbdecd234bc33e8b4"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.007648 11949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:49:28.010792 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.011945 11949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:49:28.012091 11949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "c3a71431cac04f78b11938d2d0372e02"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "568eb87e223e9b17027c4a5b155927e4"
server_key_iv: "e690269015fca5cbbdecd234bc33e8b4"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.012274 11949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:28.035990 11949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:28.039567 11949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:28.039816 11949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:28.040652 11949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:28.041934 11949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:28.042011 11949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.042127 11949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:28.042186 11949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.071235 11949 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:40349
I20260810 18:49:28.071298 12149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:40349 every 8 connection(s)
I20260810 18:49:28.072362 11949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:28.076501 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 11949
I20260810 18:49:28.076661 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:28.077257 30395 external_mini_cluster.cc:1428] Setting key 7ca492540814b13d285660713f730dce
Aug 10 18:49:28 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:28.091651 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:28.074577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48057 (local address 127.29.174.254:45929)
0810 18:49:28.074971 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:28.074976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:28.075887 (+ 911us) server_negotiation.cc:407] Connection header received
0810 18:49:28.077675 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:28.077681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:28.077777 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:28.077871 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:28.079618 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.080557 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.081993 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.082533 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.082641 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:28.085783 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:28.085789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:28.085793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:28.085844 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:28.088758 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:28.089495 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:28.089501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:28.089505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:28.089645 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:28.090084 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:28.090087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:28.090089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:28.090380 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:28.090491 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:28.091243 (+ 752us) server_negotiation.cc:299] Negotiation successful
0810 18:49:28.091406 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":206,"threads_started":1}
I20260810 18:49:28.092823 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:28.075102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:48057)
0810 18:49:28.075741 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:28.075775 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:28.077429 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:28.078103 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:28.078111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:28.078810 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:28.079454 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:28.079471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.080677 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.080680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:28.081849 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:28.081861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.082706 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.082711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:28.082887 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:28.083451 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:28.083472 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:28.085571 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:28.088888 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:28.088897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:28.088910 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:28.089374 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:28.089760 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:28.089765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:28.089768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:28.089992 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:28.090500 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:28.090508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:28.090895 (+ 387us) client_negotiation.cc:769] Sending connection context
0810 18:49:28.091176 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 18:49:28.091621 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":243,"threads_started":1}
I20260810 18:49:28.094213 12150 heartbeater.cc:347] Connected to a master server at 127.29.174.254:45929
I20260810 18:49:28.094508 12150 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:28.095173 12150 heartbeater.cc:511] Master 127.29.174.254:45929 requested a full tablet report, sending...
I20260810 18:49:28.097285 11847 ts_manager.cc:194] Registered new tserver with Master: c3a71431cac04f78b11938d2d0372e02 (127.29.174.193:40349)
I20260810 18:49:28.099113 11847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:48057
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:49:28.131583 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:28.440634 12158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:28.440881 12158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:28.440997 12158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:28.448051 12158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:28.448170 12158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:28.448335 12158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:28.457787 12158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:28.458947 12158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:28.460176 12158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:28.472281 12167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:28.472395 12170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:28.472543 12176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:28.473327 12158 server_base.cc:1034] running on GCE node
I20260810 18:49:28.473879 12158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:28.474771 12158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:28.475981 12158 hybrid_clock.cc:648] HybridClock initialized: now 1786387768475933 us; error 39 us; skew 500 ppm
I20260810 18:49:28.476274 12158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:28 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387768, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:28.479310 12158 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:49:28.480751 12158 webserver.cc:460] Webserver started at http://127.29.174.194:41359/ using document root <none> and password file <none>
I20260810 18:49:28.481433 12158 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:28.481556 12158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:28.481864 12158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:28.484791 12158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "97701b956e7842ec9d1a39cccd5106cc"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "92be827b5d9c4d247340ad1174cda9a0"
server_key_iv: "bb682537552b7191cb0bd780fa61fde8"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.485592 12158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "97701b956e7842ec9d1a39cccd5106cc"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "92be827b5d9c4d247340ad1174cda9a0"
server_key_iv: "bb682537552b7191cb0bd780fa61fde8"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.491384 12158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 18:49:28.495149 12188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.496471 12158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:49:28.496634 12158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "97701b956e7842ec9d1a39cccd5106cc"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "92be827b5d9c4d247340ad1174cda9a0"
server_key_iv: "bb682537552b7191cb0bd780fa61fde8"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.496831 12158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:28.518543 12158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:28.523679 12158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:28.523929 12158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:28.524858 12158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:28.526218 12158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:28.526289 12158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.526409 12158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:28.526458 12158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.556593 12158 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:42929
I20260810 18:49:28.556641 12385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:42929 every 8 connection(s)
I20260810 18:49:28.557782 12158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:28.560909 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 12158
I20260810 18:49:28.561107 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:28.561661 30395 external_mini_cluster.cc:1428] Setting key b894a85177b6670e596a873b5ee7838a
Aug 10 18:49:28 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387768, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:28.578619 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:28.560259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:55011 (local address 127.29.174.254:45929)
0810 18:49:28.560455 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:28.560461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:28.563074 (+ 2613us) server_negotiation.cc:407] Connection header received
0810 18:49:28.565178 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:28.565183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:28.565276 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:28.565361 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:28.566852 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.567585 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.569061 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.569477 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.569583 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:28.572761 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:28.572766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:28.572769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:28.572802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:28.575424 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:28.576231 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:28.576235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:28.576237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:28.576371 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:28.576887 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:28.576891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:28.576893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:28.577199 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:28.577319 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:28.578156 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 18:49:28.578363 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:49:28.579823 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:28.560768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:55011)
0810 18:49:28.562877 (+ 2109us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:28.562939 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:28.564911 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:28.565600 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:28.565608 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:28.566104 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:28.566685 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:28.566703 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.567734 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.567739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:28.568919 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:28.568931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:28.569690 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:28.569697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:28.569915 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:28.570533 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:28.570555 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:28.572560 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:28.575558 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:28.575567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:28.575581 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:28.576098 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:28.576480 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:28.576485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:28.576488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:28.576742 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:28.577326 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:28.577333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:28.577703 (+ 370us) client_negotiation.cc:769] Sending connection context
0810 18:49:28.578048 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 18:49:28.578526 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1717,"thread_start_us":1346,"threads_started":1}
I20260810 18:49:28.581307 12386 heartbeater.cc:347] Connected to a master server at 127.29.174.254:45929
I20260810 18:49:28.581604 12386 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:28.582324 12386 heartbeater.cc:511] Master 127.29.174.254:45929 requested a full tablet report, sending...
I20260810 18:49:28.583777 11847 ts_manager.cc:194] Registered new tserver with Master: 97701b956e7842ec9d1a39cccd5106cc (127.29.174.194:42929)
I20260810 18:49:28.584659 11847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:55011
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:49:28.623093 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:28.936373 12394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:28.936609 12394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:28.936728 12394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:28.943616 12394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:28.943709 12394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:28.943861 12394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:28.952893 12394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:28.954003 12394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:28.955178 12394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:28.967291 12401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:28.967233 12403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:28.967260 12405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:28.967990 12394 server_base.cc:1034] running on GCE node
I20260810 18:49:28.968513 12394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:28.969329 12394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:28.970507 12394 hybrid_clock.cc:648] HybridClock initialized: now 1786387768970456 us; error 40 us; skew 500 ppm
I20260810 18:49:28.970789 12394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:28 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387768, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:28.973537 12394 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:49:28.974877 12394 webserver.cc:460] Webserver started at http://127.29.174.195:33663/ using document root <none> and password file <none>
I20260810 18:49:28.975466 12394 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:28.975577 12394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:28.975857 12394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:28.978521 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "c7f746d952214f3d98ee603db4c18a5a"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "5dda760b580c9f90a9d08a7d7e4bbd87"
server_key_iv: "82bbe4d51cacd739de783f057a005738"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.979250 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "c7f746d952214f3d98ee603db4c18a5a"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "5dda760b580c9f90a9d08a7d7e4bbd87"
server_key_iv: "82bbe4d51cacd739de783f057a005738"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.983851 12394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:28.986956 12420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:28.988077 12394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:28.988212 12394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "c7f746d952214f3d98ee603db4c18a5a"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "5dda760b580c9f90a9d08a7d7e4bbd87"
server_key_iv: "82bbe4d51cacd739de783f057a005738"
server_key_version: "encryptionkey@0"
I20260810 18:49:28.988374 12394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:29.002388 12394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:29.005735 12394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:29.005971 12394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:29.006878 12394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:29.008040 12394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:29.008116 12394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:29.008224 12394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:29.008282 12394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:29.037420 12394 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37357
I20260810 18:49:29.037464 12602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37357 every 8 connection(s)
I20260810 18:49:29.038543 12394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:29.045051 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 12394
I20260810 18:49:29.045204 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:29.045616 30395 external_mini_cluster.cc:1428] Setting key 77f05c217226b5ba83faa057546197ad
Aug 10 18:49:29 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387768, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:29.055912 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:29.040721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35441 (local address 127.29.174.254:45929)
0810 18:49:29.040886 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:29.040891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:29.041963 (+ 1072us) server_negotiation.cc:407] Connection header received
0810 18:49:29.043832 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:29.043838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:29.043919 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:29.044005 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:29.045406 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.046248 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.047741 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.048184 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.048299 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:29.051331 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:29.051336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:29.051339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:29.051371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:29.053602 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.054199 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.054203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.054205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.054319 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.054666 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.054669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.054671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.054916 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:29.055021 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:29.055523 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 18:49:29.055672 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:49:29.056653 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:29.041128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:35441)
0810 18:49:29.041817 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:29.041855 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:29.043549 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:29.044230 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:29.044239 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:29.044693 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:29.045231 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:29.045254 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.046402 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.046407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:29.047609 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:29.047621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.048396 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.048403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:29.048635 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:29.049132 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:29.049152 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:29.051156 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:29.053725 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:29.053731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:29.053741 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:29.054112 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:29.054418 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:29.054421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:29.054423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:29.054584 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:29.055001 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:29.055006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:29.055265 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 18:49:29.055455 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:49:29.055775 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260810 18:49:29.057802 12603 heartbeater.cc:347] Connected to a master server at 127.29.174.254:45929
I20260810 18:49:29.058061 12603 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:29.058732 12603 heartbeater.cc:511] Master 127.29.174.254:45929 requested a full tablet report, sending...
I20260810 18:49:29.060128 11846 ts_manager.cc:194] Registered new tserver with Master: c7f746d952214f3d98ee603db4c18a5a (127.29.174.195:37357)
I20260810 18:49:29.060952 11846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:35441
I20260810 18:49:29.072227 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.193@KRBTEST.COM" created.
I20260810 18:49:29.101770 12150 heartbeater.cc:503] Master 127.29.174.254:45929 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
Entry for principal oryx/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab.
WARNING: no policy specified for oryx/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
Entry for principal oryx/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab.
WARNING: no policy specified for oryx/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
Entry for principal oryx/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab.
I20260810 18:49:29.216753 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 11949
Aug 10 18:49:29 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:49:29.235001 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:29.217823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.29.174.193:40349)
0810 18:49:29.218619 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:29.218627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:29.218655 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:49:29.218717 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:29.218724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:29.218934 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:29.219071 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:29.220310 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.221313 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.222569 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.223124 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.223222 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:29.227855 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:29.227879 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:29.227905 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:29.227989 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:29.231679 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.232558 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.232566 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.232568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.232709 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.233417 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.233421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.233424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.233801 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:29.234055 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:29.234433 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:49:29.234637 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":474,"threads_started":1}
I20260810 18:49:29.236191 12145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:29.317304 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 11949
I20260810 18:49:29.337015 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12158
Aug 10 18:49:29 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:49:29.351786 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:29.337883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.29.174.194:42929)
0810 18:49:29.338412 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:29.338419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:29.338444 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:49:29.338504 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:29.338511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:29.338711 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:29.338834 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:29.340026 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.340978 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.342298 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.342942 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.343038 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:29.346895 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:29.346906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:29.346919 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:29.346950 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:29.349502 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.350086 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.350091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.350094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.350202 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.350518 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.350521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.350523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.350839 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:29.351090 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:29.351293 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 18:49:29.351476 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":235,"threads_started":1}
I20260810 18:49:29.352959 12375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:29.433027 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 12158
I20260810 18:49:29.452430 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12394
Aug 10 18:49:29 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:49:29.467317 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:29.453370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.29.174.195:37357)
0810 18:49:29.453980 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:29.453987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:29.454015 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:49:29.454071 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:29.454078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:29.454282 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:29.454443 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:29.455609 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.456603 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.457746 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:29.458264 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:29.458490 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:29.462431 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:29.462442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:29.462455 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:29.462485 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:29.465074 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.465617 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.465622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.465625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.465750 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:29.466015 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:29.466018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:29.466020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:29.466313 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:29.466588 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:29.466800 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 18:49:29.466983 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":291,"threads_started":1}
I20260810 18:49:29.468477 12595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:29.547780 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 12394
I20260810 18:49:29.567860 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 11798
I20260810 18:49:29.568969 11898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:29.668939 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 11798
I20260810 18:49:29.693363 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45929
--webserver_interface=127.29.174.254
--webserver_port=35615
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:30.008919 12625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:30.009186 12625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:30.009272 12625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:30.016103 12625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:30.016191 12625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:30.016242 12625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:30.016295 12625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:30.016337 12625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:30.025590 12625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=35615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:30.026814 12625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:30.027976 12625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:30.038672 12638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:30.038725 12635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:30.038757 12634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:30.039743 12625 server_base.cc:1034] running on GCE node
I20260810 18:49:30.040416 12625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:30.041669 12625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:30.042860 12625 hybrid_clock.cc:648] HybridClock initialized: now 1786387770042812 us; error 36 us; skew 500 ppm
I20260810 18:49:30.043159 12625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387770, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:30.045996 12625 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:49:30.047315 12625 webserver.cc:460] Webserver started at http://127.29.174.254:35615/ using document root <none> and password file <none>
I20260810 18:49:30.047926 12625 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:30.048053 12625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:30.052989 12625 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 18:49:30.055541 12645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:30.056701 12625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:30.056856 12625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "d099cf0cf73845b1b0a755fb50cf943e"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "ae156949d6a498ad2abdb8d8798de5d7"
server_key_iv: "2506df0b068988d805eaa250daa50aeb"
server_key_version: "encryptionkey@0"
I20260810 18:49:30.057753 12625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:30.089286 12625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:30.093267 12625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:30.093499 12625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:30.110092 12625 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:45929
I20260810 18:49:30.110111 12729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:45929 every 8 connection(s)
I20260810 18:49:30.111474 12625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:30.116415 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 12625
I20260810 18:49:30.117163 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:40349
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:45929
--webserver_port=39511
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:49:30.118072 12732 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:49:30.121153 12732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Bootstrap starting.
I20260810 18:49:30.139593 12732 log.cc:824] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:30.147954 12732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:49:30.148432 12732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Bootstrap complete.
I20260810 18:49:30.157054 12732 raft_consensus.cc:348] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:30.157814 12732 raft_consensus.cc:729] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d099cf0cf73845b1b0a755fb50cf943e, State: Initialized, Role: FOLLOWER
I20260810 18:49:30.158417 12732 consensus_queue.cc:260] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:30.158639 12732 raft_consensus.cc:388] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:30.158751 12732 raft_consensus.cc:482] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:30.158912 12732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:49:30.162578 12732 raft_consensus.cc:504] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:30.163010 12732 leader_election.cc:302] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d099cf0cf73845b1b0a755fb50cf943e; no voters:
I20260810 18:49:30.163563 12732 leader_election.cc:288] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:49:30.163707 12738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:49:30.164753 12738 raft_consensus.cc:686] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [term 2 LEADER]: Becoming Leader. State: Replica: d099cf0cf73845b1b0a755fb50cf943e, State: Running, Role: LEADER
I20260810 18:49:30.165330 12738 consensus_queue.cc:237] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } }
I20260810 18:49:30.165587 12732 sys_catalog.cc:564] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:30.167768 12740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d099cf0cf73845b1b0a755fb50cf943e. Latest consensus state: current_term: 2 leader_uuid: "d099cf0cf73845b1b0a755fb50cf943e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } } }
I20260810 18:49:30.168176 12740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:30.168571 12739 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d099cf0cf73845b1b0a755fb50cf943e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099cf0cf73845b1b0a755fb50cf943e" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45929 } } }
I20260810 18:49:30.168895 12739 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:30.182271 12747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:30.191501 12747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:30.192967 12747 catalog_manager.cc:1259] Loaded cluster ID: 9dcca827321e447492e10d6e123fbb01
I20260810 18:49:30.193116 12747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:30.197463 12747 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:30.198958 12747 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d099cf0cf73845b1b0a755fb50cf943e: Loaded TSK: 0
I20260810 18:49:30.200546 12747 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:49:30.558207 12734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:30.558557 12734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:30.558710 12734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:30.570598 12734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:30.570744 12734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:30.570940 12734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:30.586798 12734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.193.keytab
--principal=oryx/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:40349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=39511
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:30.588460 12734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:30.590160 12734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:30.607795 12773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:30.607815 12774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:30.608635 12779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:30.608987 12734 server_base.cc:1034] running on GCE node
I20260810 18:49:30.609575 12734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:30.610683 12734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:30.611932 12734 hybrid_clock.cc:648] HybridClock initialized: now 1786387770611888 us; error 34 us; skew 500 ppm
I20260810 18:49:30.612259 12734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387770, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:30.615125 12734 init.cc:377] Logged in from keytab as oryx/127.29.174.193@KRBTEST.COM (short username oryx)
I20260810 18:49:30.616433 12734 webserver.cc:460] Webserver started at http://127.29.174.193:39511/ using document root <none> and password file <none>
I20260810 18:49:30.617028 12734 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:30.617161 12734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:30.622059 12734 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:49:30.624506 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:30.625670 12734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:49:30.625820 12734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "c3a71431cac04f78b11938d2d0372e02"
format_stamp: "Formatted at 2026-08-10 18:49:27 on dist-test-slave-6zbq"
server_key: "568eb87e223e9b17027c4a5b155927e4"
server_key_iv: "e690269015fca5cbbdecd234bc33e8b4"
server_key_version: "encryptionkey@0"
I20260810 18:49:30.626770 12734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:30.646596 12734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:30.650204 12734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:30.650472 12734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:30.651350 12734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:30.652624 12734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:30.652727 12734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:30.652832 12734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:30.652905 12734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:30.682749 12734 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:40349
I20260810 18:49:30.682827 12972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:40349 every 8 connection(s)
I20260810 18:49:30.684073 12734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:30.693357 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 12734
I20260810 18:49:30.693914 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:42929
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:45929
--webserver_port=41359
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:30.700439 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:30.686782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:46571)
0810 18:49:30.687514 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:30.687573 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:30.689205 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:30.690376 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:30.690387 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:30.690887 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:30.691393 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.691410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.692887 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.692891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.694144 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.694157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.695436 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.695441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.695616 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.696200 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:30.696229 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:30.699630 (+ 3401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":296,"threads_started":1}
W20260810 18:49:30.700832 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:30.686655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:46571 (local address 127.29.174.254:45929)
0810 18:49:30.687226 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:30.687238 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:30.687677 (+ 439us) server_negotiation.cc:407] Connection header received
0810 18:49:30.689636 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:30.689648 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:30.689952 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:30.690217 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:30.691551 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.692709 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.694280 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.695238 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.695424 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.700019 (+ 4595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:46571: BlockingRecv error: recv got EOF from 127.29.174.193:46571 (error 108)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":221,"threads_started":1}
W20260810 18:49:30.701062 12975 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:45929 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.174.254:45929: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:30.708774 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:30.701614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:38925)
0810 18:49:30.701776 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:30.701791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:30.701882 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:30.702255 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:30.702258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:30.702579 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:30.702708 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.702715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.703841 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.703844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.704670 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.704680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.705339 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.705345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.705511 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.705987 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:30.706003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:30.708521 (+ 2518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:49:30.708930 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:30.701753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:38925 (local address 127.29.174.254:45929)
0810 18:49:30.701896 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:30.701901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:30.701919 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:49:30.701968 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:30.701973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:30.702101 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:30.702195 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:30.702838 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.703715 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.704807 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.705203 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.705316 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.708723 (+ 3407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:38925: BlockingRecv error: recv got EOF from 127.29.174.193:38925 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:30 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:30.716087 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:30.709318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:43355)
0810 18:49:30.709437 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:30.709453 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:30.709530 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:30.709867 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:30.709872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:30.710139 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:30.710278 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.710286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.711210 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.711215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.712074 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:30.712085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.712704 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.712709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:30.712864 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.713313 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:30.713329 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:30.715826 (+ 2497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 18:49:30.716259 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:30.709482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:43355 (local address 127.29.174.254:45929)
0810 18:49:30.709640 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:30.709644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:30.709657 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:49:30.709691 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:30.709694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:30.709752 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:30.709828 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:30.710414 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.711078 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.712200 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:30.712578 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:30.712678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:30.716021 (+ 3343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:43355: BlockingRecv error: recv got EOF from 127.29.174.193:43355 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:49:30.716364 12975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:49:31.010289 12980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:31.010561 12980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:31.010694 12980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:31.017503 12980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:31.017592 12980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:31.017736 12980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:31.027026 12980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.194.keytab
--principal=oryx/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:42929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=41359
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:31.028081 12980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:31.029302 12980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:31.041090 12989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:31.041185 12988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:31.041392 12993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:31.041718 12980 server_base.cc:1034] running on GCE node
I20260810 18:49:31.042264 12980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:31.043097 12980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:31.044284 12980 hybrid_clock.cc:648] HybridClock initialized: now 1786387771044239 us; error 40 us; skew 500 ppm
I20260810 18:49:31.044555 12980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387771, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:31.047362 12980 init.cc:377] Logged in from keytab as oryx/127.29.174.194@KRBTEST.COM (short username oryx)
I20260810 18:49:31.048705 12980 webserver.cc:460] Webserver started at http://127.29.174.194:41359/ using document root <none> and password file <none>
I20260810 18:49:31.049316 12980 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:31.049437 12980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:31.054250 12980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:49:31.056628 13004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.057785 12980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:31.057924 12980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "97701b956e7842ec9d1a39cccd5106cc"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "92be827b5d9c4d247340ad1174cda9a0"
server_key_iv: "bb682537552b7191cb0bd780fa61fde8"
server_key_version: "encryptionkey@0"
I20260810 18:49:31.058838 12980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:31.097388 12980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:31.100793 12980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:31.101014 12980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:31.101876 12980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:31.103108 12980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:31.103209 12980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.103294 12980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:31.103350 12980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.132688 12980 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:42929
I20260810 18:49:31.132790 13187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:42929 every 8 connection(s)
I20260810 18:49:31.134048 12980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:31.139452 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 12980
I20260810 18:49:31.140067 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:37357
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:45929
--webserver_port=33663
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:36069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:49:31.149104 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.136480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:35411 (local address 127.29.174.254:45929)
0810 18:49:31.136659 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.136664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.137756 (+ 1092us) server_negotiation.cc:407] Connection header received
0810 18:49:31.139538 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.139544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.139625 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.139721 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.141193 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.141894 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.143631 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.144059 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.144171 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.148919 (+ 4748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:35411: BlockingRecv error: recv got EOF from 127.29.174.194:35411 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:49:31.149874 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.136938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:35411)
0810 18:49:31.137612 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.137650 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.139287 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.139950 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.139957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.140496 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.141017 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.141034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.142045 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.142049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.143483 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.143496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.144281 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.144289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.144491 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.145089 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.145110 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.148766 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":255,"threads_started":1}
W20260810 18:49:31.150501 13188 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:45929 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.174.254:45929: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:31.158443 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.151044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:56743)
0810 18:49:31.151233 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.151252 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.151341 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.151597 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.151600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.151878 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.152004 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.152011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.153262 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.153266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.154100 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.154110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.154957 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.154961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.155087 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.155607 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.155623 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.158221 (+ 2598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:49:31.159049 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.151111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:56743 (local address 127.29.174.254:45929)
0810 18:49:31.151313 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.151319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.151339 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:49:31.151423 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.151428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.151494 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.151581 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.152133 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.153123 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.154234 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.154841 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.154955 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.158834 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:56743: BlockingRecv error: recv got EOF from 127.29.174.194:56743 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:31.167557 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.159039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:39517)
0810 18:49:31.159260 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.159277 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.159364 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.159725 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.159730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.159994 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.160123 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.160130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.161162 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.161168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.162654 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.162671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.163453 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.163461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.163662 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.164189 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.164211 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.167282 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260810 18:49:31.167647 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.159146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:39517 (local address 127.29.174.254:45929)
0810 18:49:31.159309 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.159314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.159332 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:49:31.159512 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.159517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.159589 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.159673 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.160257 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.160996 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.162807 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.163286 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.163410 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.167453 (+ 4043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:39517: BlockingRecv error: recv got EOF from 127.29.174.194:39517 (error 108)
Metrics: {"server-negotiator.queue_time_us":62,"thread_start_us":285,"threads_started":1}
W20260810 18:49:31.168031 13188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:49:31.463694 13191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:31.463946 13191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:31.464036 13191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:31.471103 13191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:31.471216 13191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:31.471328 13191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:31.481182 13191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/krb5kdc/oryx_127.29.174.195.keytab
--principal=oryx/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:37357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=33663
--tserver_master_addrs=127.29.174.254:45929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:31.482502 13191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:31.483950 13191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:31.497455 13200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:31.497510 13205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:31.497484 13201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:31.498231 13191 server_base.cc:1034] running on GCE node
I20260810 18:49:31.498876 13191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:31.499823 13191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:31.501027 13191 hybrid_clock.cc:648] HybridClock initialized: now 1786387771500972 us; error 53 us; skew 500 ppm
I20260810 18:49:31.501339 13191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387771, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:31.504498 13191 init.cc:377] Logged in from keytab as oryx/127.29.174.195@KRBTEST.COM (short username oryx)
I20260810 18:49:31.506121 13191 webserver.cc:460] Webserver started at http://127.29.174.195:33663/ using document root <none> and password file <none>
I20260810 18:49:31.506912 13191 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:31.507053 13191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:31.512439 13191 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:49:31.515012 13216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.516263 13191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:49:31.516414 13191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "c7f746d952214f3d98ee603db4c18a5a"
format_stamp: "Formatted at 2026-08-10 18:49:28 on dist-test-slave-6zbq"
server_key: "5dda760b580c9f90a9d08a7d7e4bbd87"
server_key_iv: "82bbe4d51cacd739de783f057a005738"
server_key_version: "encryptionkey@0"
I20260810 18:49:31.517380 13191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:31.530754 13191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:31.534552 13191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:31.534785 13191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:31.535663 13191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:31.537042 13191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:31.537137 13191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.537240 13191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:31.537315 13191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:31.567257 13191 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37357
I20260810 18:49:31.567317 13401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37357 every 8 connection(s)
I20260810 18:49:31.568600 13191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:31.578249 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 13191
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:49:31.584838 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.571017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:51957 (local address 127.29.174.254:45929)
0810 18:49:31.571590 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.571596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.572695 (+ 1099us) server_negotiation.cc:407] Connection header received
0810 18:49:31.574759 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.574764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.574848 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.574928 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.576326 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.577028 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.578774 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.579310 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.579444 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.584552 (+ 5108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:51957: BlockingRecv error: recv got EOF from 127.29.174.195:51957 (error 108)
Metrics: {"server-negotiator.queue_time_us":416}
I20260810 18:49:31.585471 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.571547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:51957)
0810 18:49:31.572531 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.572582 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.574491 (+ 1909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.575109 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.575116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.575625 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.576163 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.576181 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.577212 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.577216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.578622 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.578635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.579684 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.579690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.579944 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.580637 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.580660 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.584345 (+ 3685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":342,"threads_started":1}
W20260810 18:49:31.586061 13402 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:45929 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.174.254:45929: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:31.593660 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.579367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.29.174.254:45929)
0810 18:49:31.579546 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.579551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.579785 (+ 234us) server_negotiation.cc:407] Connection header received
0810 18:49:31.579900 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.579905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.579975 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.580086 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.581489 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.582326 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.583677 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.584259 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.584378 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.587191 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:31.587204 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:31.587223 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:31.587274 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:31.590658 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:31.591272 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:31.591279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:31.591285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:31.591461 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:31.591804 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:31.591808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:31.591810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:31.592296 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:31.592584 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:31.592969 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 18:49:31.593325 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:49:31.594415 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.586593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:35577)
0810 18:49:31.586752 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.586769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.586838 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.587103 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.587108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.587492 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.587657 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.587667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.588686 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.588691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.589984 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.589999 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.590755 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.590761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.590911 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.591348 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.591364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.594157 (+ 2793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:49:31.594537 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.586680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35577 (local address 127.29.174.254:45929)
0810 18:49:31.586837 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.586843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.586861 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:49:31.586911 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.586915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.586976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.587063 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.587782 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.588551 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.590122 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.590622 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.590707 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.594305 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:35577: BlockingRecv error: recv got EOF from 127.29.174.195:35577 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:31.601747 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.595057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:58257)
0810 18:49:31.595196 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.595213 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.595301 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.595586 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.595590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.595909 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.596055 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.596066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.597044 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.597049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.598057 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.598069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.598718 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.598729 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.598869 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.599252 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.599266 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.601495 (+ 2229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 18:49:31.602007 13402 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:49:31.602027 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.595176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:58257 (local address 127.29.174.254:45929)
0810 18:49:31.595364 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.595368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.595381 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:49:31.595428 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.595431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.595475 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.595541 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.596181 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.596909 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.598170 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.598613 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.598703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.601704 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:58257: BlockingRecv error: recv got EOF from 127.29.174.195:58257 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:31 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:31.725512 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:31.716969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:46831)
0810 18:49:31.717442 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:31.717462 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:31.717592 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:31.717923 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:31.717927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:31.718428 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:31.718666 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.718678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.719863 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.719869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.720925 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:31.720939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.721778 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.721784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:31.721940 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.722532 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:31.722550 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:31.725186 (+ 2636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":260,"threads_started":1}
W20260810 18:49:31.725927 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:31.717017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:46831 (local address 127.29.174.254:45929)
0810 18:49:31.717192 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:31.717200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:31.717541 (+ 341us) server_negotiation.cc:407] Connection header received
0810 18:49:31.717699 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:31.717704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:31.717774 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:31.717854 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:31.718818 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.719683 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.721095 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:31.721596 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:31.721739 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:31.725412 (+ 3673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:46831: BlockingRecv error: recv got EOF from 127.29.174.193:46831 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:32.176754 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:32.168553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:37197)
0810 18:49:32.168993 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:32.169009 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:32.169120 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:32.169391 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:32.169394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:32.169831 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:32.170044 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.170058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.171275 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.171281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.172324 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.172339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.173052 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.173059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.173229 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.173786 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:32.173804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:32.176460 (+ 2656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
W20260810 18:49:32.176909 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:32.168601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:37197 (local address 127.29.174.254:45929)
0810 18:49:32.168829 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:32.168835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:32.169080 (+ 245us) server_negotiation.cc:407] Connection header received
0810 18:49:32.169208 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:32.169212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:32.169278 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:32.169351 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:32.170166 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.171112 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.172466 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.172874 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.172981 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.176665 (+ 3684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:37197: BlockingRecv error: recv got EOF from 127.29.174.194:37197 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:32.616891 13410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:32.602834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:55199)
0810 18:49:32.603324 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:32.603344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:32.603491 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:32.603863 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:32.603868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:32.604479 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:32.604735 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.604749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.605863 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.605869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.606878 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.606890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.607778 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.607785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.607950 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.613084 (+ 5134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:32.613105 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:32.616586 (+ 3481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
W20260810 18:49:32.617062 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:32.602893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:55199 (local address 127.29.174.254:45929)
0810 18:49:32.603108 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:32.603118 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:32.603422 (+ 304us) server_negotiation.cc:407] Connection header received
0810 18:49:32.603624 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:32.603630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:32.603717 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:32.603820 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:32.604893 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.605730 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.607012 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.607587 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.607710 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.616830 (+ 9120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:55199: BlockingRecv error: recv got EOF from 127.29.174.195:55199 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:32 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:32.735443 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:32.726570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:60273)
0810 18:49:32.727113 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:32.727153 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:32.727287 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:32.727598 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:32.727602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:32.728020 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:32.728380 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.728398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.729506 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.729510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.730537 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:32.730554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.731335 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.731341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:32.731489 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.732066 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:32.732086 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:32.735202 (+ 3116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":287,"threads_started":1}
W20260810 18:49:32.735651 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:32.726612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:60273 (local address 127.29.174.254:45929)
0810 18:49:32.726776 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:32.726781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:32.727278 (+ 497us) server_negotiation.cc:407] Connection header received
0810 18:49:32.727392 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:32.727396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:32.727468 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:32.727554 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:32.728580 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.729348 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.730682 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:32.731149 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:32.731250 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:32.735414 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:60273: BlockingRecv error: recv got EOF from 127.29.174.193:60273 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:33.185330 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:33.177614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:49531)
0810 18:49:33.178192 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:33.178208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:33.178321 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:33.178610 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:33.178613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:33.178998 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:33.179170 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.179179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.180086 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.180091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.180982 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.180994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.181606 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.181610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.181716 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.182222 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:33.182239 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:33.185009 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":325,"threads_started":1}
W20260810 18:49:33.185418 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:33.177723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:49531 (local address 127.29.174.254:45929)
0810 18:49:33.177881 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:33.177886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:33.178272 (+ 386us) server_negotiation.cc:407] Connection header received
0810 18:49:33.178425 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:33.178429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:33.178491 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:33.178570 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:33.179282 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.179977 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.181097 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.181489 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.181577 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.185241 (+ 3664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:49531: BlockingRecv error: recv got EOF from 127.29.174.194:49531 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:33.627583 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:33.617972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:44643)
0810 18:49:33.618504 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:33.618520 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:33.618626 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:33.618901 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:33.618903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:33.619284 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:33.619453 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.619462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.620470 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.620475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.621823 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.621838 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.622627 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.622633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.622796 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.623449 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:33.623473 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:33.627206 (+ 3733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":282,"threads_started":1}
W20260810 18:49:33.631172 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:33.618097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:44643 (local address 127.29.174.254:45929)
0810 18:49:33.618266 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:33.618271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:33.618580 (+ 309us) server_negotiation.cc:407] Connection header received
0810 18:49:33.618714 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:33.618718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:33.618795 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:33.618876 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:33.619580 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.620339 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.621982 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.622469 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.622571 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.630964 (+ 8393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:44643: BlockingRecv error: recv got EOF from 127.29.174.195:44643 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:33 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:33.744477 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:33.736276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:51799)
0810 18:49:33.736795 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:33.736832 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:33.736965 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:33.737328 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:33.737333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:33.737691 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:33.737884 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.737894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.738975 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.738980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.739888 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:33.739900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.740578 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.740584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:33.740749 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.741361 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:33.741381 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:33.744130 (+ 2749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":258,"threads_started":1}
W20260810 18:49:33.744637 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:33.736402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:51799 (local address 127.29.174.254:45929)
0810 18:49:33.736634 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:33.736641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:33.736914 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:49:33.737103 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:33.737107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:33.737173 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:33.737275 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:33.738038 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.738837 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.740030 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:33.740439 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:33.740548 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:33.744384 (+ 3836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:51799: BlockingRecv error: recv got EOF from 127.29.174.193:51799 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:34.194403 13421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:34.186251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:58263)
0810 18:49:34.186767 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:34.186784 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:34.186892 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:34.187154 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:34.187157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:34.187556 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:34.187737 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.187747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.188794 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.188797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.189668 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.189678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.190392 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.190399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.190591 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.191257 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:34.191282 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:34.194094 (+ 2812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":270,"threads_started":1}
W20260810 18:49:34.194521 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:34.186367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:58263 (local address 127.29.174.254:45929)
0810 18:49:34.186536 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:34.186541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:34.186848 (+ 307us) server_negotiation.cc:407] Connection header received
0810 18:49:34.186978 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:34.186981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:34.187044 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:34.187124 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:34.187898 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.188694 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.189770 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.190194 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.190319 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.194296 (+ 3977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:58263: BlockingRecv error: recv got EOF from 127.29.174.194:58263 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:34.638134 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:34.629973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:53985)
0810 18:49:34.630508 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:34.630523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:34.630666 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:34.631014 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:34.631018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:34.631387 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:34.631564 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.631573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.632544 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.632568 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.633707 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.633725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.634497 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.634503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.634677 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.635271 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:34.635289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:34.637879 (+ 2590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":269,"threads_started":1}
W20260810 18:49:34.638264 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:34.630031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:53985 (local address 127.29.174.254:45929)
0810 18:49:34.630224 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:34.630231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:34.630623 (+ 392us) server_negotiation.cc:407] Connection header received
0810 18:49:34.630789 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:34.630795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:34.630887 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:34.630965 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:34.631715 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.632416 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.633848 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.634332 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.634476 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.638073 (+ 3597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:53985: BlockingRecv error: recv got EOF from 127.29.174.195:53985 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:34 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:34.753207 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:34.745455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:36971)
0810 18:49:34.745912 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:34.745932 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:34.746050 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:34.746322 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:34.746325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:34.746732 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:34.746886 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.746895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.747816 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.747819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.748776 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:34.748788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.749456 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.749460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:34.749580 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.750206 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:34.750224 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:34.752980 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":242,"threads_started":1}
W20260810 18:49:34.753397 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:34.745591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:36971 (local address 127.29.174.254:45929)
0810 18:49:34.745826 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:34.745832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:34.746005 (+ 173us) server_negotiation.cc:407] Connection header received
0810 18:49:34.746143 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:34.746147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:34.746212 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:34.746291 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:34.747010 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.747717 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.748909 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:34.749332 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:34.749462 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:34.753171 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:36971: BlockingRecv error: recv got EOF from 127.29.174.193:36971 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:35.203030 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:35.195303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:48991)
0810 18:49:35.195762 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:35.195778 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:35.195892 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:35.196181 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:35.196184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:35.196566 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:35.196721 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.196730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.197631 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.197634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.198644 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.198656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.199328 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.199333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.199467 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.199995 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:35.200012 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:35.202726 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
W20260810 18:49:35.203197 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:35.195348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:48991 (local address 127.29.174.254:45929)
0810 18:49:35.195530 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:35.195537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:35.195854 (+ 317us) server_negotiation.cc:407] Connection header received
0810 18:49:35.195980 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:35.195984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:35.196049 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:35.196124 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:35.196829 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.197538 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.198766 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.199180 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.199269 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.202951 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:48991: BlockingRecv error: recv got EOF from 127.29.174.194:48991 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:35.646871 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:35.639048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:33001)
0810 18:49:35.639519 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:35.639545 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:35.639686 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:35.639971 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:35.639974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:35.640340 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:35.640490 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.640498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.641515 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.641519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.642500 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.642513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.643218 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.643224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.643395 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.643965 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:35.643983 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:35.646555 (+ 2572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":249,"threads_started":1}
W20260810 18:49:35.647020 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:35.639230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:33001 (local address 127.29.174.254:45929)
0810 18:49:35.639418 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:35.639424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:35.639630 (+ 206us) server_negotiation.cc:407] Connection header received
0810 18:49:35.639784 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:35.639788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:35.639865 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:35.639950 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:35.640620 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.641388 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.642645 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.643052 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.643154 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.646785 (+ 3631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:33001: BlockingRecv error: recv got EOF from 127.29.174.195:33001 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:35 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:35.761997 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:35.754106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:41083)
0810 18:49:35.754725 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:35.754746 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:35.754849 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:35.755172 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:35.755176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:35.755629 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:35.755800 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.755810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.756780 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.756784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.757688 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:35.757699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.758399 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.758403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:35.758526 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.759082 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:35.759101 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:35.761723 (+ 2622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":241,"threads_started":1}
W20260810 18:49:35.762081 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:35.754167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:41083 (local address 127.29.174.254:45929)
0810 18:49:35.754416 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:35.754422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:35.754816 (+ 394us) server_negotiation.cc:407] Connection header received
0810 18:49:35.754950 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:35.754956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:35.755031 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:35.755121 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:35.755929 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.756663 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.757836 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:35.758255 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:35.758372 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:35.761912 (+ 3540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:41083: BlockingRecv error: recv got EOF from 127.29.174.193:41083 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:36.212594 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:36.203918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:44891)
0810 18:49:36.204378 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:36.204394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:36.204498 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:36.204782 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:36.204785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:36.205210 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:36.205429 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.205444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.206735 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.206740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.207641 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.207654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.208667 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.208679 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.208812 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.209537 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:36.209560 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:36.212338 (+ 2778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":224,"threads_started":1}
W20260810 18:49:36.212733 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:36.204068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:44891 (local address 127.29.174.254:45929)
0810 18:49:36.204240 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:36.204244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:36.204455 (+ 211us) server_negotiation.cc:407] Connection header received
0810 18:49:36.204590 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:36.204594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:36.204667 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:36.204739 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:36.205579 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.206614 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.207797 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.208409 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.208518 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.212514 (+ 3996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:44891: BlockingRecv error: recv got EOF from 127.29.174.194:44891 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:36.655685 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:36.647712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:33391)
0810 18:49:36.648210 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:36.648225 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:36.648349 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:36.648673 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:36.648677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:36.649096 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:36.649252 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.649261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.650239 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.650243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.651156 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.651168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.651865 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.651870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.652035 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.652578 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:36.652597 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:36.655445 (+ 2848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":233,"threads_started":1}
W20260810 18:49:36.655833 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:36.647817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:33391 (local address 127.29.174.254:45929)
0810 18:49:36.648007 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:36.648014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:36.648308 (+ 294us) server_negotiation.cc:407] Connection header received
0810 18:49:36.648460 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:36.648464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:36.648543 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:36.648627 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:36.649399 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.650103 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.651292 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.651728 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.651839 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.655618 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:33391: BlockingRecv error: recv got EOF from 127.29.174.195:33391 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:36 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:36.771060 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:36.763055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:50941)
0810 18:49:36.763576 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:36.763597 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:36.763708 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:36.763977 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:36.763980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:36.764359 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:36.764517 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.764526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.765497 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.765501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.766462 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:36.766475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.767159 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.767163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:36.767296 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.767787 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:36.767803 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:36.770772 (+ 2969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":281,"threads_started":1}
W20260810 18:49:36.771179 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:36.763114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:50941 (local address 127.29.174.254:45929)
0810 18:49:36.763273 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:36.763277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:36.763657 (+ 380us) server_negotiation.cc:407] Connection header received
0810 18:49:36.763812 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:36.763816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:36.763877 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:36.763948 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:36.764637 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.765392 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.766567 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:36.767027 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:36.767129 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:36.770951 (+ 3822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:50941: BlockingRecv error: recv got EOF from 127.29.174.193:50941 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:37.221240 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:37.213441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:42985)
0810 18:49:37.213924 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:37.213940 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:37.214051 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:37.214321 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:37.214324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:37.214721 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:37.214870 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.214879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.215792 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.215795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.216667 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.216678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.217331 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.217336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.217462 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.218154 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:37.218178 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:37.220979 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":275,"threads_started":1}
W20260810 18:49:37.221609 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:37.213520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:42985 (local address 127.29.174.254:45929)
0810 18:49:37.213696 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:37.213703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:37.214004 (+ 301us) server_negotiation.cc:407] Connection header received
0810 18:49:37.214143 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:37.214147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:37.214217 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:37.214292 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:37.214980 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.215680 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.216792 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.217192 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.217289 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.221231 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:42985: BlockingRecv error: recv got EOF from 127.29.174.194:42985 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:37.668090 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:37.656545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:38647)
0810 18:49:37.657053 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:37.657068 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:37.657194 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:37.657465 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:37.657468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:37.657830 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:37.657984 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.657993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.659020 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.659023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.659902 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.659913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.660638 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.660642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.660777 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.664957 (+ 4180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:37.664976 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:37.667804 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":276,"threads_started":1}
W20260810 18:49:37.668265 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:37.656621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:38647 (local address 127.29.174.254:45929)
0810 18:49:37.656809 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:37.656815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:37.657158 (+ 343us) server_negotiation.cc:407] Connection header received
0810 18:49:37.657281 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:37.657285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:37.657363 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:37.657440 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:37.658088 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.658905 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.660045 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.660523 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.660618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.668017 (+ 7399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:38647: BlockingRecv error: recv got EOF from 127.29.174.195:38647 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:37 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:37.779845 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:37.771893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:32801)
0810 18:49:37.772472 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:37.772491 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:37.772635 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:37.772968 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:37.772973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:37.773480 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:37.773701 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.773715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.774856 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.774860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.775771 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:37.775781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.776371 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.776375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:37.776485 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.776982 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:37.776999 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:37.779588 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":247,"threads_started":1}
W20260810 18:49:37.780011 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:37.771983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:32801 (local address 127.29.174.254:45929)
0810 18:49:37.772166 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:37.772171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:37.772559 (+ 388us) server_negotiation.cc:407] Connection header received
0810 18:49:37.772741 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:37.772746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:37.772830 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:37.772925 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:37.773863 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.774727 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.775873 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:37.776257 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:37.776347 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:37.779822 (+ 3475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:32801: BlockingRecv error: recv got EOF from 127.29.174.193:32801 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:38.230247 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:38.222129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:48617)
0810 18:49:38.222661 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:38.222678 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:38.222800 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:38.223067 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:38.223070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:38.223444 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:38.223605 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.223615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.224602 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.224606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.225591 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.225606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.226398 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.226404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.226531 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.227217 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:38.227242 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:38.229991 (+ 2749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":240,"threads_started":1}
W20260810 18:49:38.230450 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:38.222174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:48617 (local address 127.29.174.254:45929)
0810 18:49:38.222398 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:38.222405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:38.222753 (+ 348us) server_negotiation.cc:407] Connection header received
0810 18:49:38.222882 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:38.222885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:38.222964 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:38.223040 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:38.223722 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.224440 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.225708 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.226226 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.226331 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.230193 (+ 3862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:48617: BlockingRecv error: recv got EOF from 127.29.174.194:48617 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:38.676846 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:38.668972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:54975)
0810 18:49:38.669512 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:38.669528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:38.669658 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:38.669971 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:38.669975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:38.670429 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:38.670600 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.670610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.671568 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.671572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.672466 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.672478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.673171 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.673185 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.673334 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.673854 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:38.673871 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:38.676588 (+ 2717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":243,"threads_started":1}
W20260810 18:49:38.676964 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:38.669073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:54975 (local address 127.29.174.254:45929)
0810 18:49:38.669266 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:38.669272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:38.669607 (+ 335us) server_negotiation.cc:407] Connection header received
0810 18:49:38.669768 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:38.669772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:38.669847 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:38.669925 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:38.670744 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.671433 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.672603 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.673029 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.673136 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.676777 (+ 3641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:54975: BlockingRecv error: recv got EOF from 127.29.174.195:54975 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:38 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:38.788456 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:38.780750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:58317)
0810 18:49:38.781294 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:38.781315 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:38.781429 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:38.781692 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:38.781695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:38.782080 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:38.782242 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.782250 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.783223 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.783227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.784134 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:38.784145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.784968 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.784975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:38.785113 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.785656 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:38.785680 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:38.788242 (+ 2562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
W20260810 18:49:38.788650 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:38.780905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:58317 (local address 127.29.174.254:45929)
0810 18:49:38.781061 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:38.781065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:38.781380 (+ 315us) server_negotiation.cc:407] Connection header received
0810 18:49:38.781510 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:38.781514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:38.781576 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:38.781661 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:38.782378 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.783102 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.784252 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:38.784759 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:38.784882 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:38.788422 (+ 3540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:58317: BlockingRecv error: recv got EOF from 127.29.174.193:58317 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:39.239205 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:39.231116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:37375)
0810 18:49:39.231593 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:39.231609 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:39.231731 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:39.231990 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:39.231993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:39.232360 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:39.232505 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.232514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.233666 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.233669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.234824 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.234841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.235621 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.235627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.235746 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.236252 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:39.236268 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:39.238876 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":246,"threads_started":1}
W20260810 18:49:39.239307 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:39.231209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:37375 (local address 127.29.174.254:45929)
0810 18:49:39.231430 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:39.231436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:39.231685 (+ 249us) server_negotiation.cc:407] Connection header received
0810 18:49:39.231820 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:39.231824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:39.231888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:39.231963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:39.232612 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.233560 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.234952 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.235448 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.235568 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.239098 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:37375: BlockingRecv error: recv got EOF from 127.29.174.194:37375 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:39.685712 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:39.677815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:60699)
0810 18:49:39.678300 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:39.678317 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:39.678450 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:39.678773 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:39.678776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:39.679156 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:39.679309 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.679317 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.680401 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.680405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.681346 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.681359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.682077 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.682081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.682184 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.682747 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:39.682768 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:39.685405 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
W20260810 18:49:39.685853 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:39.677857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:60699 (local address 127.29.174.254:45929)
0810 18:49:39.678050 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:39.678055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:39.678430 (+ 375us) server_negotiation.cc:407] Connection header received
0810 18:49:39.678536 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:39.678540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:39.678651 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:39.678741 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:39.679491 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.680237 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.681500 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.681935 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.682028 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.685598 (+ 3570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:60699: BlockingRecv error: recv got EOF from 127.29.174.195:60699 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:39 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:39.797573 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:39.789452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:56439)
0810 18:49:39.789965 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:39.789985 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:39.790099 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:39.790376 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:39.790379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:39.790762 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:39.790940 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.790949 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.791941 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.791946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.793181 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:39.793195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.793867 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.793871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:39.793986 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.794522 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:39.794539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:39.797249 (+ 2710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":264,"threads_started":1}
W20260810 18:49:39.797711 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:39.789495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:56439 (local address 127.29.174.254:45929)
0810 18:49:39.789673 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:39.789678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:39.790049 (+ 371us) server_negotiation.cc:407] Connection header received
0810 18:49:39.790185 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:39.790189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:39.790253 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:39.790372 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:39.791047 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.791807 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.793316 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:39.793749 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:39.793846 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:39.797496 (+ 3650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:56439: BlockingRecv error: recv got EOF from 127.29.174.193:56439 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:40.247951 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:40.240096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:60223)
0810 18:49:40.240601 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:40.240617 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:40.240729 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:40.241041 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:40.241044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:40.241454 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:40.241622 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.241631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.242594 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.242599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.243521 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.243533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.244279 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.244286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.244434 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.245011 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:40.245029 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:40.247719 (+ 2690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
W20260810 18:49:40.248075 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:40.240208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:60223 (local address 127.29.174.254:45929)
0810 18:49:40.240386 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:40.240391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:40.240682 (+ 291us) server_negotiation.cc:407] Connection header received
0810 18:49:40.240865 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:40.240869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:40.240935 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:40.241014 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:40.241737 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.242490 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.243660 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.244117 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.244222 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.247883 (+ 3661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:60223: BlockingRecv error: recv got EOF from 127.29.174.194:60223 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:40.693969 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:40.686752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:39103)
0810 18:49:40.687207 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:40.687223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:40.687329 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:40.687609 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:40.687612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:40.687985 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:40.688149 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.688158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.689043 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.689047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.689956 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.689967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.690646 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.690650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.690769 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.691240 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:40.691256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:40.693748 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":249,"threads_started":1}
W20260810 18:49:40.694126 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:40.686790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:39103 (local address 127.29.174.254:45929)
0810 18:49:40.686981 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:40.686988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:40.687299 (+ 311us) server_negotiation.cc:407] Connection header received
0810 18:49:40.687416 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:40.687420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:40.687497 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:40.687572 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:40.688268 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.688949 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.690058 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.690508 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.690591 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.693925 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:39103: BlockingRecv error: recv got EOF from 127.29.174.195:39103 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:40 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:40.806435 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:40.798503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:37713)
0810 18:49:40.798970 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:40.798990 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:40.799119 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:40.799423 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:40.799428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:40.799890 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:40.800035 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.800044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.801134 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.801140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.802095 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:40.802110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.802902 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.802908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:40.803063 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.803623 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:40.803641 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:40.806178 (+ 2537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":272,"threads_started":1}
W20260810 18:49:40.806632 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:40.798583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:37713 (local address 127.29.174.254:45929)
0810 18:49:40.798753 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:40.798759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:40.799080 (+ 321us) server_negotiation.cc:407] Connection header received
0810 18:49:40.799230 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:40.799234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:40.799296 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:40.799375 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:40.800181 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.800991 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.802249 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:40.802748 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:40.802860 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:40.806417 (+ 3557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:37713: BlockingRecv error: recv got EOF from 127.29.174.193:37713 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:41.256842 13455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:41.248780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:39963)
0810 18:49:41.249215 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:41.249231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:41.249339 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:41.249600 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:41.249603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:41.249953 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:41.250115 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.250125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.251084 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.251089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.252078 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.252091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.252987 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.252992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.253187 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.253889 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:41.253909 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:41.256591 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":228,"threads_started":1}
W20260810 18:49:41.257004 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:41.248844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:39963 (local address 127.29.174.254:45929)
0810 18:49:41.249069 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:41.249075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:41.249303 (+ 228us) server_negotiation.cc:407] Connection header received
0810 18:49:41.249425 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:41.249429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:41.249493 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:41.249568 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:41.250238 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.250968 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.252211 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.252813 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.252948 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.256777 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:39963: BlockingRecv error: recv got EOF from 127.29.174.194:39963 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:41.702744 13456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:41.694833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:34063)
0810 18:49:41.695325 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:41.695341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:41.695445 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:41.695700 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:41.695703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:41.696046 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:41.696186 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.696194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.697115 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.697119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.698045 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.698056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.698851 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.698855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.698974 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.699523 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:41.699539 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:41.702437 (+ 2898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":278,"threads_started":1}
W20260810 18:49:41.702826 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:41.694967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:34063 (local address 127.29.174.254:45929)
0810 18:49:41.695126 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:41.695131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:41.695404 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:49:41.695521 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:41.695525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:41.695597 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:41.695674 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:41.696311 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.697003 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.698210 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.698726 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.698836 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.702616 (+ 3780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:34063: BlockingRecv error: recv got EOF from 127.29.174.195:34063 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:41 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:41.814747 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:41.807303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:33051)
0810 18:49:41.807772 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:41.807793 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:41.807907 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:41.808184 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:41.808187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:41.808577 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:41.808773 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.808783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.809694 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.809697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.810624 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:41.810636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.811258 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.811261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:41.811378 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.811887 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:41.811903 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:41.814503 (+ 2600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":233,"threads_started":1}
W20260810 18:49:41.814924 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:41.807403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:33051 (local address 127.29.174.254:45929)
0810 18:49:41.807581 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:41.807586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:41.807856 (+ 270us) server_negotiation.cc:407] Connection header received
0810 18:49:41.807990 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:41.807993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:41.808072 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:41.808154 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:41.808878 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.809595 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.810733 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:41.811140 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:41.811238 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:41.814702 (+ 3464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:33051: BlockingRecv error: recv got EOF from 127.29.174.193:33051 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:42.266633 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:42.257669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:37083)
0810 18:49:42.258117 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:42.258150 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:42.258269 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:42.258586 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:42.258589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:42.258973 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:42.259173 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.259184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.260437 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.260445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.261850 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.261873 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.262743 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.262750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.262916 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.263540 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:42.263558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:42.266375 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":227,"threads_started":1}
W20260810 18:49:42.266774 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:42.257722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:37083 (local address 127.29.174.254:45929)
0810 18:49:42.257934 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:42.257940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:42.258254 (+ 314us) server_negotiation.cc:407] Connection header received
0810 18:49:42.258392 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:42.258397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:42.258466 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:42.258547 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:42.259343 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.260108 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.262057 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.262545 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.262659 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.266570 (+ 3911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:37083: BlockingRecv error: recv got EOF from 127.29.174.194:37083 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:42.714964 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:42.703632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:44801)
0810 18:49:42.704147 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:42.704162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:42.704272 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:42.704544 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:42.704547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:42.705020 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:42.705201 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.705210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.706471 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.706475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.707340 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.707351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.707967 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.707971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.708085 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.712130 (+ 4045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:42.712147 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:42.714732 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":270,"threads_started":1}
W20260810 18:49:42.715098 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:42.703723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:44801 (local address 127.29.174.254:45929)
0810 18:49:42.703890 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:42.703894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:42.704225 (+ 331us) server_negotiation.cc:407] Connection header received
0810 18:49:42.704356 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:42.704360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:42.704435 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:42.704533 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:42.705346 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.706364 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.707448 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.707863 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.707946 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.714901 (+ 6955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:44801: BlockingRecv error: recv got EOF from 127.29.174.195:44801 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:42 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:42.824914 13462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:42.815607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:42077)
0810 18:49:42.816105 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:42.816125 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:42.816265 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:42.816572 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:42.816578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:42.816975 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:42.817129 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.817138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.818128 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.818132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.819068 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:42.819080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.819787 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.819792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:42.819958 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.820553 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:42.820572 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:42.824670 (+ 4098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
W20260810 18:49:42.825078 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:42.815677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:42077 (local address 127.29.174.254:45929)
0810 18:49:42.815835 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:42.815841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:42.816216 (+ 375us) server_negotiation.cc:407] Connection header received
0810 18:49:42.816373 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:42.816377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:42.816440 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:42.816519 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:42.817274 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.817991 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.819203 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:42.819650 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:42.819761 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:42.824878 (+ 5117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:42077: BlockingRecv error: recv got EOF from 127.29.174.193:42077 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:43.275661 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:43.267520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:48591)
0810 18:49:43.268047 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:43.268062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:43.268287 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:43.268774 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:43.268779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:43.269201 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:43.269358 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.269367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.270406 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.270411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.271342 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.271354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.271999 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.272003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.272135 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.272653 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:43.272670 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:43.275434 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":300,"threads_started":1}
W20260810 18:49:43.275836 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:43.267591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:48591 (local address 127.29.174.254:45929)
0810 18:49:43.267828 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:43.267835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:43.268221 (+ 386us) server_negotiation.cc:407] Connection header received
0810 18:49:43.268536 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:43.268543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:43.268624 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:43.268719 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:43.269493 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.270211 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.271469 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.271891 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.271984 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.275637 (+ 3653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:48591: BlockingRecv error: recv got EOF from 127.29.174.194:48591 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:43.723380 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:43.715784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:38627)
0810 18:49:43.716284 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:43.716301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:43.716409 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:43.716704 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:43.716707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:43.717091 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:43.717257 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.717266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.718239 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.718242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.719138 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.719149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.719758 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.719762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.719885 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.720458 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:43.720476 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:43.723107 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":275,"threads_started":1}
W20260810 18:49:43.723572 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:43.715842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:38627 (local address 127.29.174.254:45929)
0810 18:49:43.716025 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:43.716030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:43.716365 (+ 335us) server_negotiation.cc:407] Connection header received
0810 18:49:43.716495 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:43.716499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:43.716575 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:43.716670 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:43.717401 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.718117 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.719244 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.719657 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.719745 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.723330 (+ 3585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:38627: BlockingRecv error: recv got EOF from 127.29.174.195:38627 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:43 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:43.833918 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:43.825776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:38373)
0810 18:49:43.826303 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:43.826322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:43.826474 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:43.826751 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:43.826754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:43.827108 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:43.827282 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.827291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.828391 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.828397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.829552 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:43.829564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.830304 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.830310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:43.830496 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.830999 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:43.831015 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:43.833684 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":255,"threads_started":1}
W20260810 18:49:43.834079 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:43.825873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:38373 (local address 127.29.174.254:45929)
0810 18:49:43.826050 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:43.826055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:43.826421 (+ 366us) server_negotiation.cc:407] Connection header received
0810 18:49:43.826577 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:43.826581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:43.826653 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:43.826723 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:43.827421 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.828253 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.829682 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:43.830113 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:43.830254 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:43.833873 (+ 3619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:38373: BlockingRecv error: recv got EOF from 127.29.174.193:38373 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:44.284631 13470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:44.276476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:40871)
0810 18:49:44.276981 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:44.276996 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:44.277120 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:44.277431 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:44.277435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:44.277847 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:44.278002 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.278012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.279242 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.279246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.280277 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.280292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.281022 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.281027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.281182 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.281770 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:44.281790 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:44.284401 (+ 2611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":254,"threads_started":1}
W20260810 18:49:44.284819 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:44.276654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:40871 (local address 127.29.174.254:45929)
0810 18:49:44.276846 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:44.276852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:44.277065 (+ 213us) server_negotiation.cc:407] Connection header received
0810 18:49:44.277245 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:44.277249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:44.277312 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:44.277390 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:44.278138 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.279100 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.280406 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.280820 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.280913 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.284590 (+ 3677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:40871: BlockingRecv error: recv got EOF from 127.29.174.194:40871 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:44.732398 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:44.724200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:34805)
0810 18:49:44.724652 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:44.724667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:44.724792 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:44.725179 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:44.725188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:44.725625 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:44.725805 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.725815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.726892 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.726896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.727809 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.727821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.728541 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.728546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.728663 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.729186 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:44.729204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:44.732076 (+ 2872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":223,"threads_started":1}
W20260810 18:49:44.732515 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:44.724263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:34805 (local address 127.29.174.254:45929)
0810 18:49:44.724452 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:44.724458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:44.724735 (+ 277us) server_negotiation.cc:407] Connection header received
0810 18:49:44.724930 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:44.724935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:44.725002 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:44.725092 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:44.725951 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.726756 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.727938 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.728381 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.728472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.732299 (+ 3827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:34805: BlockingRecv error: recv got EOF from 127.29.174.195:34805 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:44 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:44.842614 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:44.834719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:58475)
0810 18:49:44.835161 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:44.835181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:44.835291 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:44.835541 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:44.835544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:44.835917 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:44.836136 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.836151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.837299 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.837304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.838198 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:44.838210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.838937 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.838942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:44.839095 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.839634 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:44.839650 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:44.842254 (+ 2604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":234,"threads_started":1}
W20260810 18:49:44.842726 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:44.834821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:58475 (local address 127.29.174.254:45929)
0810 18:49:44.835049 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:44.835054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:44.835240 (+ 186us) server_negotiation.cc:407] Connection header received
0810 18:49:44.835373 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:44.835377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:44.835440 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:44.835516 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:44.836286 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.837166 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.838335 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:44.838798 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:44.838910 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:44.842503 (+ 3593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:58475: BlockingRecv error: recv got EOF from 127.29.174.193:58475 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:45.293360 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:45.285463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:34997)
0810 18:49:45.285939 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:45.285955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:45.286067 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:45.286324 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:45.286327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:45.286705 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:45.286878 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.286887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.287838 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.287842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.288829 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.288844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.289681 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.289694 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.289831 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.290423 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:45.290440 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:45.293025 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":239,"threads_started":1}
W20260810 18:49:45.293452 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:45.285538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:34997 (local address 127.29.174.254:45929)
0810 18:49:45.285691 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:45.285695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:45.286022 (+ 327us) server_negotiation.cc:407] Connection header received
0810 18:49:45.286154 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:45.286158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:45.286220 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:45.286305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:45.286989 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.287728 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.288960 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.289536 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.289653 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.293230 (+ 3577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:34997: BlockingRecv error: recv got EOF from 127.29.174.194:34997 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.195@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:45.741325 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:45.733337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.195:41553)
0810 18:49:45.733811 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:45.733826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:45.733957 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:45.734301 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:45.734306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:45.734731 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:45.734922 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.734932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.735917 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.735921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.736834 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.736847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.737538 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.737544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.737710 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.738312 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:45.738331 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:45.741055 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
W20260810 18:49:45.741506 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:45.733421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:41553 (local address 127.29.174.254:45929)
0810 18:49:45.733615 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:45.733622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:45.733904 (+ 282us) server_negotiation.cc:407] Connection header received
0810 18:49:45.734082 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:45.734086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:45.734164 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:45.734245 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:45.735064 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.735770 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.736972 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.737398 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.737509 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.741308 (+ 3799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.195:41553: BlockingRecv error: recv got EOF from 127.29.174.195:41553 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:45 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.193@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:45.851405 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:45.843709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.193:45681)
0810 18:49:45.844146 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:45.844166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:45.844280 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:45.844534 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:45.844537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:45.844927 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:45.845091 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.845115 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.846058 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.846061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.846982 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:45.846993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.847614 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.847618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:45.847722 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.848231 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:45.848247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:45.851042 (+ 2795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":220,"threads_started":1}
W20260810 18:49:45.851503 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:45.843780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:45681 (local address 127.29.174.254:45929)
0810 18:49:45.844013 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:45.844019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:45.844238 (+ 219us) server_negotiation.cc:407] Connection header received
0810 18:49:45.844363 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:45.844367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:45.844432 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:45.844509 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:45.845224 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.845953 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.847092 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:45.847512 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:45.847602 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:45.851269 (+ 3667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.193:45681: BlockingRecv error: recv got EOF from 127.29.174.193:45681 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 18:49:46 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:49:46 dist-test-slave-6zbq krb5kdc[11777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.174.194@KRBTEST.COM for oryx/127.29.174.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:49:46.302114 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:46.294441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45929 (local address 127.29.174.194:49335)
0810 18:49:46.294907 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:46.294923 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:46.295028 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:46.295298 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:46.295301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:46.295674 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:46.295843 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:46.295853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.296773 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.296776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:46.297675 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:46.297686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.298303 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.298308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:46.298466 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:46.298995 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:46.299011 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:46.301776 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.174.254:45929: Server oryx/127.29.174.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":259,"threads_started":1}
W20260810 18:49:46.302248 12977 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:46.294497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:49335 (local address 127.29.174.254:45929)
0810 18:49:46.294692 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:46.294700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:46.294994 (+ 294us) server_negotiation.cc:407] Connection header received
0810 18:49:46.295115 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:46.295118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:46.295194 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:46.295271 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:46.295948 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.296663 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.297775 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.298202 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.298289 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:46.301970 (+ 3681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.174.194:49335: BlockingRecv error: recv got EOF from 127.29.174.194:49335 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:49:46.588893 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12734
I20260810 18:49:46.599881 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:46.589843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.29.174.193:40349)
0810 18:49:46.590403 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:46.590413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:46.590442 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:49:46.590499 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:46.590505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:46.590714 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:46.590861 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:46.592187 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.593162 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.594242 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.594825 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.594916 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:46.597113 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:46.597121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:46.597132 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:46.597161 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:46.599420 (+ 2259us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:49:46.599656 (+ 236us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42986: Request ticket server kudu/127.29.174.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":271,"threads_started":1}
W20260810 18:49:46.599978 13487 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42986: Request ticket server kudu/127.29.174.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:49:46.600008 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12734: Not authorized: Client connection negotiation failed: client connection to 127.29.174.193:40349: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:49:46.600216 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 12734
I20260810 18:49:46.629282 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12980
I20260810 18:49:46.639815 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:46.630045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.29.174.194:42929)
0810 18:49:46.630594 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:46.630600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:46.630629 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:49:46.630685 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:46.630691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:46.630895 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:46.631027 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:46.632143 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.633169 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.634254 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.634893 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.634990 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:46.637131 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:46.637139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:46.637151 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:46.637187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:46.639360 (+ 2173us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:49:46.639586 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49968: Request ticket server kudu/127.29.174.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":231,"threads_started":1}
W20260810 18:49:46.639905 13492 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49968: Request ticket server kudu/127.29.174.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:49:46.639931 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12980: Not authorized: Client connection negotiation failed: client connection to 127.29.174.194:42929: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:49:46.640074 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 12980
I20260810 18:49:46.661916 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 13191
I20260810 18:49:46.672458 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:46.662684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.29.174.195:37357)
0810 18:49:46.663229 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:46.663235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:46.663261 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:49:46.663318 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:46.663324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:46.663520 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:46.663664 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:46.664817 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.665847 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.667082 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:46.667599 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:46.667700 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:46.669814 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:46.669822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:46.669833 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:46.669868 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:46.672009 (+ 2141us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:49:46.672235 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55216: Request ticket server kudu/127.29.174.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":261,"threads_started":1}
W20260810 18:49:46.672518 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 13191: Not authorized: Client connection negotiation failed: client connection to 127.29.174.195:37357: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.29.174.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:49:46.672540 13493 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55216: Request ticket server kudu/127.29.174.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:49:46.672673 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 13191
I20260810 18:49:46.691556 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 12625
I20260810 18:49:46.692667 12724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:46.806684 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 12625
2026-08-10T18:49:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21204 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T18:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:46Z Disabled control of system clock
I20260810 18:49:46.892484 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36499
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:49:47.203323 13500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:47.203563 13500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:47.203678 13500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:47.210484 13500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:47.210575 13500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:47.210638 13500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:47.210714 13500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:47.210774 13500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:47.219293 13500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:36499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:47.220292 13500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:47.221871 13500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:47.232585 13515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:47.232554 13511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:47.232666 13510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:47.233482 13500 server_base.cc:1034] running on GCE node
I20260810 18:49:47.234123 13500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:47.235370 13500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:47.236554 13500 hybrid_clock.cc:648] HybridClock initialized: now 1786387787236518 us; error 30 us; skew 500 ppm
I20260810 18:49:47.236857 13500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:47.238938 13500 webserver.cc:460] Webserver started at http://127.29.174.254:33487/ using document root <none> and password file <none>
I20260810 18:49:47.239544 13500 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:47.239634 13500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:47.239933 13500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:47.242579 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "a51e8cf5fc214f69b1acec207774850f"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "b177d4c442a2cca6867fde21bc8be5b0"
server_key_iv: "bf86e89996fa1561e4317eca686e15e6"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.243307 13500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "a51e8cf5fc214f69b1acec207774850f"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "b177d4c442a2cca6867fde21bc8be5b0"
server_key_iv: "bf86e89996fa1561e4317eca686e15e6"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.247947 13500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:49:47.251039 13523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:47.252218 13500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:47.252375 13500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "a51e8cf5fc214f69b1acec207774850f"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "b177d4c442a2cca6867fde21bc8be5b0"
server_key_iv: "bf86e89996fa1561e4317eca686e15e6"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.252558 13500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:47.285912 13500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:47.286686 13500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:47.286931 13500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:47.303642 13500 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:36499
I20260810 18:49:47.303660 13600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:36499 every 8 connection(s)
I20260810 18:49:47.304884 13500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:47.308341 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:47.310256 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 13500
I20260810 18:49:47.310451 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:47.311072 30395 external_mini_cluster.cc:1428] Setting key 9b5dfeee6888e68cac55f40b96a1cf9a
I20260810 18:49:47.320624 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f: Bootstrap starting.
I20260810 18:49:47.324316 13603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:47.325574 13603 log.cc:824] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:47.325868 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:47.313037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.29.174.254:36499)
0810 18:49:47.314051 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:47.314064 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:47.314113 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:49:47.316383 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:47.316409 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:47.316978 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:47.317381 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:47.317892 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.319814 (+ 1922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.321098 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.322075 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.322260 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:47.323312 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:47.323324 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:47.323342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:47.323993 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:47.324117 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:47.324156 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:47.324278 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:47.324412 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 18:49:47.324708 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":312,"threads_started":1}
I20260810 18:49:47.328888 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f: No bootstrap required, opened a new log
I20260810 18:49:47.335726 13603 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } }
I20260810 18:49:47.336040 13603 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:47.336123 13603 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51e8cf5fc214f69b1acec207774850f, State: Initialized, Role: FOLLOWER
I20260810 18:49:47.336633 13603 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } }
I20260810 18:49:47.336855 13603 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:47.336977 13603 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:47.337129 13603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:47.339159 13603 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } }
I20260810 18:49:47.339607 13603 leader_election.cc:302] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51e8cf5fc214f69b1acec207774850f; no voters:
I20260810 18:49:47.340168 13603 leader_election.cc:288] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:47.340272 13614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:47.340878 13614 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [term 1 LEADER]: Becoming Leader. State: Replica: a51e8cf5fc214f69b1acec207774850f, State: Running, Role: LEADER
I20260810 18:49:47.341410 13614 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } }
I20260810 18:49:47.341961 13603 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:47.343747 13615 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51e8cf5fc214f69b1acec207774850f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } } }
I20260810 18:49:47.343827 13617 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51e8cf5fc214f69b1acec207774850f. Latest consensus state: current_term: 1 leader_uuid: "a51e8cf5fc214f69b1acec207774850f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51e8cf5fc214f69b1acec207774850f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36499 } } }
I20260810 18:49:47.344131 13615 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:47.344136 13617 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:47.346221 13624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:47.351238 13624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:47.360170 13624 catalog_manager.cc:1347] Generated new cluster ID: 012848d392ae4146a0c2ab849522310f
I20260810 18:49:47.360278 13624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:47.376474 13624 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:47.377666 13624 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:47.383824 13624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51e8cf5fc214f69b1acec207774850f: Generated new TSK 0
I20260810 18:49:47.384768 13624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:47.399399 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--builtin_ntp_servers=127.29.174.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 18:49:47.726621 13643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:47.726881 13643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:47.726990 13643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:47.735328 13643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:47.735426 13643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:47.735601 13643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:47.744737 13643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:47.745986 13643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:47.748309 13643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:47.760871 13653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:47.760825 13654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:47.760905 13661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:47.762104 13643 server_base.cc:1034] running on GCE node
I20260810 18:49:47.762720 13643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:47.763600 13643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:47.764832 13643 hybrid_clock.cc:648] HybridClock initialized: now 1786387787764778 us; error 40 us; skew 500 ppm
I20260810 18:49:47.765148 13643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:47.771195 13643 webserver.cc:460] Webserver started at http://127.29.174.193:46517/ using document root <none> and password file <none>
I20260810 18:49:47.771871 13643 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:47.771976 13643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:47.772297 13643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:47.775391 13643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "058b2ed5e7b3406a9717f3194401b7e5"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "d304774e3ffe04556b89d3a6f86d9379"
server_key_iv: "6d8453f3107be6d67149793b47e5ad24"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.776188 13643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "058b2ed5e7b3406a9717f3194401b7e5"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "d304774e3ffe04556b89d3a6f86d9379"
server_key_iv: "6d8453f3107be6d67149793b47e5ad24"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.781193 13643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:49:47.784472 13670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:47.785624 13643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:47.785766 13643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "058b2ed5e7b3406a9717f3194401b7e5"
format_stamp: "Formatted at 2026-08-10 18:49:47 on dist-test-slave-6zbq"
server_key: "d304774e3ffe04556b89d3a6f86d9379"
server_key_iv: "6d8453f3107be6d67149793b47e5ad24"
server_key_version: "encryptionkey@0"
I20260810 18:49:47.785951 13643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:47.809250 13643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:47.810034 13643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:47.810312 13643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:47.811224 13643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:47.812606 13643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:47.812701 13643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:47.812815 13643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:47.812885 13643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:47.843755 13643 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:39201
I20260810 18:49:47.843839 13859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:39201 every 8 connection(s)
I20260810 18:49:47.844972 13643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:47.846668 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 13643
I20260810 18:49:47.846843 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:47.847667 30395 external_mini_cluster.cc:1428] Setting key f92e5d6415d42e7f41a3f98cd247b953
I20260810 18:49:47.851205 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--builtin_ntp_servers=127.29.174.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:49:47.858383 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:47.847596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48343 (local address 127.29.174.254:36499)
0810 18:49:47.848053 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:47.848058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:47.849737 (+ 1679us) server_negotiation.cc:407] Connection header received
0810 18:49:47.851796 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:47.851801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:47.851878 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:47.851991 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:47.853045 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.853982 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.855546 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.856067 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.856227 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:47.857549 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:47.857554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:47.857557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:47.857573 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:47.857616 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:47.857628 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:47.857719 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:47.857923 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:49:47.858018 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":233,"threads_started":1}
I20260810 18:49:47.859298 13865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:47.848336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36499 (local address 127.29.174.193:48343)
0810 18:49:47.849582 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:47.849628 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:47.851542 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:47.852228 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:47.852236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:47.852326 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:47.852878 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:47.852896 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.854105 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.854108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:47.855407 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:47.855423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:47.856306 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:47.856312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:47.856549 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:47.857215 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:47.857218 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:47.857255 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:47.857329 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:47.857724 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:47.857736 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:47.857743 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:49:47.857897 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:49:47.858269 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":346,"threads_started":1}
I20260810 18:49:47.860702 13861 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36499
I20260810 18:49:47.860965 13861 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:47.861662 13861 heartbeater.cc:511] Master 127.29.174.254:36499 requested a full tablet report, sending...
I20260810 18:49:47.863844 13543 ts_manager.cc:194] Registered new tserver with Master: 058b2ed5e7b3406a9717f3194401b7e5 (127.29.174.193:39201)
I20260810 18:49:47.865289 13543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.193:48343
W20260810 18:49:48.173838 13867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:48.174083 13867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:48.174204 13867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:48.181304 13867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:48.181391 13867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:48.181522 13867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:48.190426 13867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:48.191556 13867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:48.193254 13867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:48.205175 13879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:48.205240 13878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:48.205205 13882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:48.206211 13867 server_base.cc:1034] running on GCE node
I20260810 18:49:48.206847 13867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:48.207724 13867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:48.208916 13867 hybrid_clock.cc:648] HybridClock initialized: now 1786387788208871 us; error 34 us; skew 500 ppm
I20260810 18:49:48.209197 13867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:48.211226 13867 webserver.cc:460] Webserver started at http://127.29.174.194:39239/ using document root <none> and password file <none>
I20260810 18:49:48.211850 13867 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:48.211946 13867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:48.212261 13867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:48.215540 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "daf5018777544afaae1528c009a95816"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "36d2d772a8bf3c05e39c4b871f6b3047"
server_key_iv: "5ffe01cdc23e45690f312b032793da40"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.216375 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "daf5018777544afaae1528c009a95816"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "36d2d772a8bf3c05e39c4b871f6b3047"
server_key_iv: "5ffe01cdc23e45690f312b032793da40"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.221220 13867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 18:49:48.224316 13893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.225690 13867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:49:48.225901 13867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "daf5018777544afaae1528c009a95816"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "36d2d772a8bf3c05e39c4b871f6b3047"
server_key_iv: "5ffe01cdc23e45690f312b032793da40"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.226097 13867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:48.243508 13867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:48.244230 13867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:48.244483 13867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:48.245539 13867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:48.247217 13867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:48.247319 13867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.247428 13867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:48.247496 13867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.277410 13867 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:34043
I20260810 18:49:48.277472 14079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:34043 every 8 connection(s)
I20260810 18:49:48.278625 13867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:48.279016 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 13867
I20260810 18:49:48.279199 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:48.280400 30395 external_mini_cluster.cc:1428] Setting key 1cf8fd588295162fc9b661ad35411a6d
I20260810 18:49:48.283741 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--builtin_ntp_servers=127.29.174.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:49:48.292398 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.281286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:46457 (local address 127.29.174.254:36499)
0810 18:49:48.281473 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.281479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.282995 (+ 1516us) server_negotiation.cc:407] Connection header received
0810 18:49:48.285278 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.285282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.285345 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.285431 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.286679 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.287577 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.289597 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.290093 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.290191 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.291667 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:48.291673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:48.291677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:48.291696 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:48.291731 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:48.291744 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:48.291830 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:48.292041 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 18:49:48.292126 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:48.293614 14085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.281808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36499 (local address 127.29.174.194:46457)
0810 18:49:48.282847 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:48.282908 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:48.284990 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:48.285675 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:48.285685 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:48.285790 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:48.286516 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:48.286539 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.287737 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.287742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:48.289397 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:48.289418 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.290289 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.290296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:48.290573 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.291341 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:48.291345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:48.291404 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:48.291491 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:48.291844 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:48.291854 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:48.291860 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:49:48.292020 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:49:48.292380 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260810 18:49:48.294849 14081 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36499
I20260810 18:49:48.295101 14081 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:48.295838 14081 heartbeater.cc:511] Master 127.29.174.254:36499 requested a full tablet report, sending...
I20260810 18:49:48.297305 13543 ts_manager.cc:194] Registered new tserver with Master: daf5018777544afaae1528c009a95816 (127.29.174.194:34043)
I20260810 18:49:48.298332 13543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.194:46457
W20260810 18:49:48.614257 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:48.614542 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:48.614629 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:48.621582 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:48.621670 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:48.621781 14086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:48.631664 14086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:48.632828 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:48.634554 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:48.646654 14096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:48.646700 14097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:48.646620 14099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:48.647418 14086 server_base.cc:1034] running on GCE node
I20260810 18:49:48.647951 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:48.648765 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:48.649959 14086 hybrid_clock.cc:648] HybridClock initialized: now 1786387788649916 us; error 34 us; skew 500 ppm
I20260810 18:49:48.650249 14086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:48.652208 14086 webserver.cc:460] Webserver started at http://127.29.174.195:43177/ using document root <none> and password file <none>
I20260810 18:49:48.652848 14086 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:48.652935 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:48.653229 14086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:48.656236 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "b910f023ad114b50b72117f0d44667e8"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "1cb21ca44036aac6258024cccd6e16d5"
server_key_iv: "aabdc9c4812bf3119dbc7ebb828e07fb"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.657191 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "b910f023ad114b50b72117f0d44667e8"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "1cb21ca44036aac6258024cccd6e16d5"
server_key_iv: "aabdc9c4812bf3119dbc7ebb828e07fb"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.664884 14086 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.008s
I20260810 18:49:48.668346 14119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.669534 14086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:48.669677 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "b910f023ad114b50b72117f0d44667e8"
format_stamp: "Formatted at 2026-08-10 18:49:48 on dist-test-slave-6zbq"
server_key: "1cb21ca44036aac6258024cccd6e16d5"
server_key_iv: "aabdc9c4812bf3119dbc7ebb828e07fb"
server_key_version: "encryptionkey@0"
I20260810 18:49:48.669873 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:48.683111 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:48.683822 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:48.684072 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:48.684921 14086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:48.686198 14086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:48.686285 14086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.686404 14086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:48.686470 14086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:48.714816 14086 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34423
I20260810 18:49:48.714880 14298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34423 every 8 connection(s)
I20260810 18:49:48.716022 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:48.724375 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 14086
I20260810 18:49:48.724504 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:48.724941 30395 external_mini_cluster.cc:1428] Setting key 3698368e6a1c80ec0faa0ee6e7443cff
I20260810 18:49:48.727644 13864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.718485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:33057 (local address 127.29.174.254:36499)
0810 18:49:48.718690 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.718696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.719948 (+ 1252us) server_negotiation.cc:407] Connection header received
0810 18:49:48.721584 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.721588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.721659 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.721734 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.722713 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.723450 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.724969 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.725542 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.725664 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.726929 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:48.726936 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:48.726940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:48.726958 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:48.726994 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:48.727006 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:48.727073 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:48.727296 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 18:49:48.727371 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:49:48.728626 14301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.718975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:36499 (local address 127.29.174.195:33057)
0810 18:49:48.719805 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:48.719846 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:48.721358 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:48.721904 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:48.721911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:48.721988 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:48.722574 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:48.722592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.723558 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.723561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:48.724829 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:48.724842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.725741 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.725746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:48.725933 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.726627 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:48.726631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:48.726678 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:48.726761 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:48.727097 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:48.727107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:48.727114 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:49:48.727282 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 18:49:48.727623 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":271,"threads_started":1}
I20260810 18:49:48.729727 14299 heartbeater.cc:347] Connected to a master server at 127.29.174.254:36499
I20260810 18:49:48.729992 14299 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:48.730772 14299 heartbeater.cc:511] Master 127.29.174.254:36499 requested a full tablet report, sending...
I20260810 18:49:48.732048 13543 ts_manager.cc:194] Registered new tserver with Master: b910f023ad114b50b72117f0d44667e8 (127.29.174.195:34423)
I20260810 18:49:48.732820 13543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.195:33057
I20260810 18:49:48.739995 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:49:48.745786 13864 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:48.744389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.29.174.254:36499)
0810 18:49:48.744570 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.744576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.744974 (+ 398us) server_negotiation.cc:407] Connection header received
0810 18:49:48.745087 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.745090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.745156 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.745230 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.745568 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58060: BlockingRecv error: recv got EOF from 127.0.0.1:58060 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:48.746546 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 13643
I20260810 18:49:48.753473 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.747496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.29.174.193:39201)
0810 18:49:48.747984 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.747990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.748013 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:49:48.748066 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.748079 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.748242 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.748353 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.748676 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.749627 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.750859 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.751375 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.751476 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.752333 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:48.752339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:48.752349 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:48.752847 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:48.752887 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:48.752904 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:48.752985 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:48.753104 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 18:49:48.753201 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":260,"threads_started":1}
I20260810 18:49:48.754551 13854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:48.834506 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 13643
I20260810 18:49:48.854472 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 13867
I20260810 18:49:48.861773 14317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.855347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.29.174.194:34043)
0810 18:49:48.855863 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.855870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.855898 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:49:48.855958 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.855964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.856199 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.856369 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.856707 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.857723 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.858887 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.859488 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.859596 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.860490 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:48.860498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:48.860509 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:48.861057 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:48.861116 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:48.861135 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:48.861218 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:48.861310 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 18:49:48.861430 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":248,"threads_started":1}
I20260810 18:49:48.862989 14070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:48.942481 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 13867
I20260810 18:49:48.963474 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 14086
I20260810 18:49:48.970686 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:48.964293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.29.174.195:34423)
0810 18:49:48.964875 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:48.964881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:48.964913 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:49:48.964974 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:48.964980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:48.965200 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:48.965326 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:48.965664 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.966713 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.967893 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:48.968434 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:48.968528 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:48.969419 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:48.969427 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:48.969439 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:48.969980 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:48.970022 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:48.970040 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:48.970130 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:48.970220 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 18:49:48.970323 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":266,"threads_started":1}
I20260810 18:49:48.971956 14293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:49.050127 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 14086
I20260810 18:49:49.071728 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 13500
I20260810 18:49:49.072813 13596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:49.172022 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 13500
2026-08-10T18:49:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2358 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T18:49:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:49Z Disabled control of system clock
I20260810 18:49:49.249168 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:44313
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:44313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:49:49.557957 14328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:49.558218 14328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:49.558323 14328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:49.565696 14328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:49.565789 14328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:49.565833 14328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:49.565910 14328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:49.565969 14328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:49.574649 14328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:44313
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:44313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:49.575728 14328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:49.577386 14328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:49.588266 14335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:49.588341 14336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:49.588306 14339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:49.589159 14328 server_base.cc:1034] running on GCE node
I20260810 18:49:49.589818 14328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:49.591041 14328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:49.592275 14328 hybrid_clock.cc:648] HybridClock initialized: now 1786387789592216 us; error 45 us; skew 500 ppm
I20260810 18:49:49.592590 14328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:49.594712 14328 webserver.cc:460] Webserver started at http://127.29.174.254:46661/ using document root <none> and password file <none>
I20260810 18:49:49.595341 14328 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:49.595425 14328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:49.595717 14328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:49.598489 14328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "9610116639b94d3ea0760362d3c0f10b"
format_stamp: "Formatted at 2026-08-10 18:49:49 on dist-test-slave-6zbq"
server_key: "9e2ca5d08584bcead24b3032d396811c"
server_key_iv: "7cd845a3eeaa0dbc88d931328c39abea"
server_key_version: "encryptionkey@0"
I20260810 18:49:49.599220 14328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "9610116639b94d3ea0760362d3c0f10b"
format_stamp: "Formatted at 2026-08-10 18:49:49 on dist-test-slave-6zbq"
server_key: "9e2ca5d08584bcead24b3032d396811c"
server_key_iv: "7cd845a3eeaa0dbc88d931328c39abea"
server_key_version: "encryptionkey@0"
I20260810 18:49:49.604079 14328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:49:49.607127 14351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:49.608259 14328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:49:49.608408 14328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "9610116639b94d3ea0760362d3c0f10b"
format_stamp: "Formatted at 2026-08-10 18:49:49 on dist-test-slave-6zbq"
server_key: "9e2ca5d08584bcead24b3032d396811c"
server_key_iv: "7cd845a3eeaa0dbc88d931328c39abea"
server_key_version: "encryptionkey@0"
I20260810 18:49:49.608595 14328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:49.613901 14328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:49.614616 14328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:49.614869 14328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:49.632452 14328 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:44313
I20260810 18:49:49.632488 14441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:44313 every 8 connection(s)
I20260810 18:49:49.633574 14328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:49.636879 14442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:49.637334 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 14328
I20260810 18:49:49.637524 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:49.638109 30395 external_mini_cluster.cc:1428] Setting key b4068ffaafae96c0f8611a18f9bcab36
I20260810 18:49:49.645736 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:49.639993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.29.174.254:44313)
0810 18:49:49.640914 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:49.640927 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:49.640965 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:49:49.642550 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:49.642569 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:49.642956 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:49.643289 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:49.644239 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:49.644247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:49.644259 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:49.644280 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:49.644335 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:49.644351 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:49.644543 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:49.644709 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 18:49:49.644913 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":309,"threads_started":1}
I20260810 18:49:49.648569 14442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b: Bootstrap starting.
I20260810 18:49:49.651266 14442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:49.652170 14442 log.cc:824] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:49.654831 14442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b: No bootstrap required, opened a new log
I20260810 18:49:49.662367 14442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } }
I20260810 18:49:49.662709 14442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:49.662811 14442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9610116639b94d3ea0760362d3c0f10b, State: Initialized, Role: FOLLOWER
I20260810 18:49:49.663372 14442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } }
I20260810 18:49:49.663648 14442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:49.663833 14442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:49.664021 14442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:49.666088 14442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } }
I20260810 18:49:49.666558 14442 leader_election.cc:302] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9610116639b94d3ea0760362d3c0f10b; no voters:
I20260810 18:49:49.667069 14442 leader_election.cc:288] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:49.667193 14452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:49.667845 14452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [term 1 LEADER]: Becoming Leader. State: Replica: 9610116639b94d3ea0760362d3c0f10b, State: Running, Role: LEADER
I20260810 18:49:49.668413 14452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } }
I20260810 18:49:49.668984 14442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:49.671183 14455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9610116639b94d3ea0760362d3c0f10b. Latest consensus state: current_term: 1 leader_uuid: "9610116639b94d3ea0760362d3c0f10b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } } }
I20260810 18:49:49.671156 14454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9610116639b94d3ea0760362d3c0f10b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9610116639b94d3ea0760362d3c0f10b" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44313 } } }
I20260810 18:49:49.671563 14455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:49.671640 14454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:49.672963 14463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:49.677155 14463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:49.685215 14463 catalog_manager.cc:1347] Generated new cluster ID: 0c390740c43d4cafb462f913f8279a71
I20260810 18:49:49.685319 14463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:49.709797 14463 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:49.710268 14463 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:49.721184 14463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9610116639b94d3ea0760362d3c0f10b: Generated new TSK 0
I20260810 18:49:49.721901 14463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:49:49.734328 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--builtin_ntp_servers=127.29.174.212:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:49:50.060956 14481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:50.061225 14481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:50.061347 14481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:50.068151 14481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:50.068238 14481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:50.068344 14481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:50.076879 14481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:50.077922 14481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:50.079568 14481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:50.091805 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.091890 14489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.091857 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:50.092625 14481 server_base.cc:1034] running on GCE node
I20260810 18:49:50.093190 14481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:50.094022 14481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:50.095197 14481 hybrid_clock.cc:648] HybridClock initialized: now 1786387790095142 us; error 51 us; skew 500 ppm
I20260810 18:49:50.095476 14481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:50.097532 14481 webserver.cc:460] Webserver started at http://127.29.174.193:39673/ using document root <none> and password file <none>
I20260810 18:49:50.098162 14481 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:50.098263 14481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:50.098568 14481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:50.101341 14481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "9084014ed0e44588932a8d73e8e27c9b"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "9094070e29f08d524a834f0e469a9e72"
server_key_iv: "4e688b89ef84d3ff7388e0cf9a409ff1"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.102052 14481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "9084014ed0e44588932a8d73e8e27c9b"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "9094070e29f08d524a834f0e469a9e72"
server_key_iv: "4e688b89ef84d3ff7388e0cf9a409ff1"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.106722 14481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:49:50.109838 14500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.111053 14481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:50.111207 14481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "9084014ed0e44588932a8d73e8e27c9b"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "9094070e29f08d524a834f0e469a9e72"
server_key_iv: "4e688b89ef84d3ff7388e0cf9a409ff1"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.111423 14481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:50.116511 14481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:50.117198 14481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:50.117420 14481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:50.118258 14481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:50.119562 14481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:50.119655 14481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.119760 14481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:50.119830 14481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.147523 14481 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:45311
I20260810 18:49:50.147580 14698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:45311 every 8 connection(s)
I20260810 18:49:50.149101 14481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:50.151835 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 14481
I20260810 18:49:50.152050 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:50.152876 30395 external_mini_cluster.cc:1428] Setting key babe2d2403daa77860a965246cb0b458
I20260810 18:49:50.155550 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--builtin_ntp_servers=127.29.174.212:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:49:50.156800 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.151386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49037 (local address 127.29.174.254:44313)
0810 18:49:50.151791 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:50.151795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:50.153478 (+ 1683us) server_negotiation.cc:407] Connection header received
0810 18:49:50.154785 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:50.154790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:50.154853 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:50.154960 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:50.156121 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:50.156127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:50.156131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:50.156150 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:50.156198 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:50.156211 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:50.156331 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:50.156459 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 18:49:50.156519 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":208,"threads_started":1}
I20260810 18:49:50.157636 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.151973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:44313 (local address 127.29.174.193:49037)
0810 18:49:50.152875 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:50.152917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:50.154493 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:50.155116 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:50.155123 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:50.155203 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:50.155847 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:50.155849 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:50.155884 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:50.155962 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:50.156303 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:50.156309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:50.156313 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:49:50.156503 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:49:50.156675 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":305,"threads_started":1}
I20260810 18:49:50.159016 14700 heartbeater.cc:347] Connected to a master server at 127.29.174.254:44313
I20260810 18:49:50.159292 14700 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:50.159879 14700 heartbeater.cc:511] Master 127.29.174.254:44313 requested a full tablet report, sending...
I20260810 18:49:50.161923 14379 ts_manager.cc:194] Registered new tserver with Master: 9084014ed0e44588932a8d73e8e27c9b (127.29.174.193:45311)
W20260810 18:49:50.471886 14709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:50.472119 14709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:50.472250 14709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:50.479393 14709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:50.479493 14709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:50.479640 14709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:50.488337 14709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:50.489426 14709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:50.491104 14709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:50.502842 14720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.502977 14727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.503002 14721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:50.503866 14709 server_base.cc:1034] running on GCE node
I20260810 18:49:50.504352 14709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:50.505291 14709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:50.506496 14709 hybrid_clock.cc:648] HybridClock initialized: now 1786387790506448 us; error 41 us; skew 500 ppm
I20260810 18:49:50.506803 14709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:50.508927 14709 webserver.cc:460] Webserver started at http://127.29.174.194:37225/ using document root <none> and password file <none>
I20260810 18:49:50.509550 14709 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:50.509660 14709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:50.509958 14709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:50.512970 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "94756d301f9e45ae800f8535113d9685"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "0c31981c8d7999343aab82852cd561cb"
server_key_iv: "fcf8e11f0db02422efe3983004b6b578"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.513746 14709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "94756d301f9e45ae800f8535113d9685"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "0c31981c8d7999343aab82852cd561cb"
server_key_iv: "fcf8e11f0db02422efe3983004b6b578"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.518567 14709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 18:49:50.521628 14740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.522857 14709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:50.522993 14709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "94756d301f9e45ae800f8535113d9685"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "0c31981c8d7999343aab82852cd561cb"
server_key_iv: "fcf8e11f0db02422efe3983004b6b578"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.523178 14709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:50.528232 14709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:50.528931 14709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:50.529160 14709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:50.529986 14709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:50.531296 14709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:50.531390 14709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.531491 14709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:50.531560 14709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.560514 14709 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:37327
I20260810 18:49:50.560572 14931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:37327 every 8 connection(s)
I20260810 18:49:50.561700 14709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:50.568925 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.564046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:49427 (local address 127.29.174.254:44313)
0810 18:49:50.564255 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:50.564290 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:50.565567 (+ 1277us) server_negotiation.cc:407] Connection header received
0810 18:49:50.567134 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:50.567138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:50.567193 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:50.567292 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:50.568373 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:50.568377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:50.568380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:50.568396 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:50.568428 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:50.568439 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:50.568526 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:50.568655 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 18:49:50.568701 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:49:50.569772 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.564574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:44313 (local address 127.29.174.194:49427)
0810 18:49:50.565419 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:50.565459 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:50.566915 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:50.567445 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:50.567452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:50.567531 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:50.568100 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:50.568103 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:50.568136 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:50.568214 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:50.568510 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:50.568515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:50.568520 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:49:50.568705 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 18:49:50.568885 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":269,"threads_started":1}
I20260810 18:49:50.571039 14933 heartbeater.cc:347] Connected to a master server at 127.29.174.254:44313
I20260810 18:49:50.571214 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 14709
I20260810 18:49:50.571336 14933 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:50.571424 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:50.571967 30395 external_mini_cluster.cc:1428] Setting key 261bb236a753b31e1081a8af06ff4be1
I20260810 18:49:50.572000 14933 heartbeater.cc:511] Master 127.29.174.254:44313 requested a full tablet report, sending...
I20260810 18:49:50.573448 14377 ts_manager.cc:194] Registered new tserver with Master: 94756d301f9e45ae800f8535113d9685 (127.29.174.194:37327)
I20260810 18:49:50.574676 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--builtin_ntp_servers=127.29.174.212:36845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:49:50.893097 14940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:50.893335 14940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:50.893427 14940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:50.900235 14940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:50.900314 14940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:50.900420 14940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:50.908840 14940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:36845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:44313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:50.909869 14940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:50.911528 14940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:50.923458 14948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.923488 14956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:50.923455 14953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:50.924278 14940 server_base.cc:1034] running on GCE node
I20260810 18:49:50.924767 14940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:50.925624 14940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:50.926797 14940 hybrid_clock.cc:648] HybridClock initialized: now 1786387790926755 us; error 42 us; skew 500 ppm
I20260810 18:49:50.927065 14940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:49:50.929030 14940 webserver.cc:460] Webserver started at http://127.29.174.195:41067/ using document root <none> and password file <none>
I20260810 18:49:50.929621 14940 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:50.929713 14940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:50.930001 14940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:50.932617 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "4ae516e928ae42bea99effeac61db4a3"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "449735c4d36c8dfe5e5a4030353a69c1"
server_key_iv: "f606d98d85062afea06314dcb4debfa0"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.933351 14940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "4ae516e928ae42bea99effeac61db4a3"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "449735c4d36c8dfe5e5a4030353a69c1"
server_key_iv: "f606d98d85062afea06314dcb4debfa0"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.937911 14940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260810 18:49:50.940937 14969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.942078 14940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:49:50.942227 14940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "4ae516e928ae42bea99effeac61db4a3"
format_stamp: "Formatted at 2026-08-10 18:49:50 on dist-test-slave-6zbq"
server_key: "449735c4d36c8dfe5e5a4030353a69c1"
server_key_iv: "f606d98d85062afea06314dcb4debfa0"
server_key_version: "encryptionkey@0"
I20260810 18:49:50.942466 14940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:50.947543 14940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:50.948228 14940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:50.948455 14940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:50.949251 14940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:50.950529 14940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:50.950623 14940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.950726 14940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:50.950796 14940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:50.980616 14940 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:39877
I20260810 18:49:50.980684 15157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:39877 every 8 connection(s)
I20260810 18:49:50.981861 14940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:50.988933 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 14940
I20260810 18:49:50.989104 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:50.989444 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.984188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:34307 (local address 127.29.174.254:44313)
0810 18:49:50.984372 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:50.984376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:50.985751 (+ 1375us) server_negotiation.cc:407] Connection header received
0810 18:49:50.987402 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:50.987407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:50.987470 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:50.987557 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:50.988753 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:50.988759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:50.988764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:50.988781 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:50.988813 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:50.988825 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:50.988932 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:50.989105 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 18:49:50.989181 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:49:50.989508 30395 external_mini_cluster.cc:1428] Setting key 6ebd1feef946a7d474706a1a1f1043eb
I20260810 18:49:50.990382 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:50.984680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:44313 (local address 127.29.174.195:34307)
0810 18:49:50.985608 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:50.985643 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:50.987155 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:50.987778 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:50.987785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:50.987866 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:50.988467 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:49:50.988470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:50.988503 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:49:50.988581 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:50.988906 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:50.988917 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:50.988925 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:49:50.989181 (+ 256us) client_negotiation.cc:240] Negotiation successful
0810 18:49:50.989405 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":257,"threads_started":1}
I20260810 18:49:50.991609 15158 heartbeater.cc:347] Connected to a master server at 127.29.174.254:44313
I20260810 18:49:50.991899 15158 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:50.992497 15158 heartbeater.cc:511] Master 127.29.174.254:44313 requested a full tablet report, sending...
I20260810 18:49:50.993664 14377 ts_manager.cc:194] Registered new tserver with Master: 4ae516e928ae42bea99effeac61db4a3 (127.29.174.195:39877)
I20260810 18:49:51.004367 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:49:51.010365 14704 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:49:51.008611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58464 (local address 127.29.174.254:44313)
0810 18:49:51.008775 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:51.008780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:51.009384 (+ 604us) server_negotiation.cc:407] Connection header received
0810 18:49:51.009572 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:51.009577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:51.009655 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:51.009750 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:51.010094 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58464: BlockingRecv error: recv got EOF from 127.0.0.1:58464 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:51.010857 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 14481
I20260810 18:49:51.014236 15171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:51.011850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.29.174.193:45311)
0810 18:49:51.012429 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:51.012435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:51.012458 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:49:51.012511 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:51.012516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:51.012698 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:51.012808 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:51.013611 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:51.013617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:51.013628 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:51.013651 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:51.013690 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:51.013704 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:51.013799 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:51.013854 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 18:49:51.013925 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":275,"threads_started":1}
I20260810 18:49:51.015399 14693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:51.094678 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 14481
I20260810 18:49:51.113041 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 14709
I20260810 18:49:51.116532 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:51.113962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.29.174.194:37327)
0810 18:49:51.114464 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:51.114471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:51.114497 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:49:51.114558 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:51.114564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:51.114774 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:51.114919 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:51.115840 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:51.115847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:51.115859 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:51.115877 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:51.115916 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:51.115929 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:51.116048 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:51.116188 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 18:49:51.116287 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260810 18:49:51.117664 14924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:51.197111 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 14709
I20260810 18:49:51.218204 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 14940
I20260810 18:49:51.222038 15176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:51.219085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.29.174.195:39877)
0810 18:49:51.219731 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:51.219737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:51.219782 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:49:51.219845 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:51.219851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:51.220062 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:51.220202 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:51.221320 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:51.221327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:51.221338 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:49:51.221355 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:51.221393 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:49:51.221406 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:51.221561 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:51.221639 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 18:49:51.221741 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":314,"threads_started":1}
I20260810 18:49:51.223237 15149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:51.301783 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 14940
I20260810 18:49:51.318962 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 14328
I20260810 18:49:51.320271 14434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:51.419238 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 14328
2026-08-10T18:49:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2247 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:51 dist-test-slave-6zbq krb5kdc[15187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:51 dist-test-slave-6zbq krb5kdc[15187](info): set up 2 sockets
Aug 10 18:49:51 dist-test-slave-6zbq krb5kdc[15187](info): commencing operation
krb5kdc: starting...
W20260810 18:49:52.648118 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:52 dist-test-slave-6zbq krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:49:52.866717 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39837
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:53.188951 15211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:53.189188 15211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:53.189312 15211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:53.196113 15211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:49:53.196203 15211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:53.196278 15211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:53.196353 15211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:49:53.196412 15211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:49:53.205374 15211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:53.206463 15211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:53.207568 15211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:53.218168 15221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:53.218111 15225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:53.218168 15227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:53.219255 15211 server_base.cc:1034] running on GCE node
I20260810 18:49:53.219929 15211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:53.221144 15211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:53.222328 15211 hybrid_clock.cc:648] HybridClock initialized: now 1786387793222282 us; error 45 us; skew 500 ppm
I20260810 18:49:53.222651 15211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:53 dist-test-slave-6zbq krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387793, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:53.225478 15211 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:49:53.226819 15211 webserver.cc:460] Webserver started at http://127.29.174.254:39189/ using document root <none> and password file <none>
I20260810 18:49:53.227406 15211 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:53.227515 15211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:53.227795 15211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:53.230568 15211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "453912337f7b4421a10131e0693428f0"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "4635b063237fc9f05b2a59accde89bae"
server_key_iv: "9180fd3944213953681d31f6b947bc4c"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.231271 15211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "453912337f7b4421a10131e0693428f0"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "4635b063237fc9f05b2a59accde89bae"
server_key_iv: "9180fd3944213953681d31f6b947bc4c"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.235860 15211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 18:49:53.238870 15239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:53.240016 15211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:49:53.240192 15211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "453912337f7b4421a10131e0693428f0"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "4635b063237fc9f05b2a59accde89bae"
server_key_iv: "9180fd3944213953681d31f6b947bc4c"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.240362 15211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:53.254992 15211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:53.258482 15211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:53.258721 15211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:53.274988 15211 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39837
I20260810 18:49:53.275020 15329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39837 every 8 connection(s)
I20260810 18:49:53.276134 15211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:49:53.278728 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 15211
I20260810 18:49:53.278956 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:49:53.279744 15330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:53.279779 30395 external_mini_cluster.cc:1428] Setting key 6c1f9a490955e3da71007386e7c2b184
I20260810 18:49:53.293064 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: Bootstrap starting.
Aug 10 18:49:53 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:53.296420 15330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:53.297739 15330 log.cc:824] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:53.300987 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: No bootstrap required, opened a new log
I20260810 18:49:53.302302 15341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:53.281598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.29.174.254:39837)
0810 18:49:53.282296 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:53.282310 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:53.282376 (+ 66us) server_negotiation.cc:407] Connection header received
0810 18:49:53.284146 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:53.284167 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:53.284525 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:53.284820 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:53.286603 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.287785 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.289434 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.290085 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.290220 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:53.294529 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:53.294539 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:53.294562 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:53.294604 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:53.298089 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:53.298921 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:53.298927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:53.298933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:53.299088 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:53.299457 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:53.299462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:53.299465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:53.300108 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:53.300411 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:53.300683 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:49:53.301064 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":294,"threads_started":1}
I20260810 18:49:53.309083 15330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } }
I20260810 18:49:53.309423 15330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:53.309511 15330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453912337f7b4421a10131e0693428f0, State: Initialized, Role: FOLLOWER
I20260810 18:49:53.309995 15330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } }
I20260810 18:49:53.310199 15330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:49:53.310333 15330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:49:53.310523 15330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:53.312731 15330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } }
I20260810 18:49:53.313119 15330 leader_election.cc:302] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 453912337f7b4421a10131e0693428f0; no voters:
I20260810 18:49:53.313619 15330 leader_election.cc:288] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:49:53.313724 15343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:53.314586 15343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [term 1 LEADER]: Becoming Leader. State: Replica: 453912337f7b4421a10131e0693428f0, State: Running, Role: LEADER
I20260810 18:49:53.315044 15343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } }
I20260810 18:49:53.315348 15330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:49:53.317663 15346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 453912337f7b4421a10131e0693428f0. Latest consensus state: current_term: 1 leader_uuid: "453912337f7b4421a10131e0693428f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } } }
I20260810 18:49:53.318061 15346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:53.319317 15345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "453912337f7b4421a10131e0693428f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453912337f7b4421a10131e0693428f0" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39837 } } }
I20260810 18:49:53.319725 15345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:49:53.321702 15354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:49:53.326608 15354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:49:53.335215 15354 catalog_manager.cc:1347] Generated new cluster ID: 44eb1838cd4447afa2e3f188a2e88cf5
I20260810 18:49:53.335310 15354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:49:53.351317 15354 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:49:53.352481 15354 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:49:53.364660 15354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: Generated new TSK 0
I20260810 18:49:53.365597 15354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:49:53.434194 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:53.751462 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:53.751693 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:53.751822 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:53.758633 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:53.758725 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:53.758874 15379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:49:53.767907 15379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:53.769063 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:53.770278 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:53.782532 15392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:53.782469 15394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:53.782555 15390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:53.783149 15379 server_base.cc:1034] running on GCE node
I20260810 18:49:53.783699 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:53.784504 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:53.785727 15379 hybrid_clock.cc:648] HybridClock initialized: now 1786387793785676 us; error 34 us; skew 500 ppm
I20260810 18:49:53.786046 15379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:53 dist-test-slave-6zbq krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387793, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:53.788956 15379 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:49:53.790208 15379 webserver.cc:460] Webserver started at http://127.29.174.193:34141/ using document root <none> and password file <none>
I20260810 18:49:53.790835 15379 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:53.790956 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:53.791224 15379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:53.793902 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "e1251748d48f4fedb51b8d6c18320d56"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "6b8eb857eb550f17e74e5668ecf1b568"
server_key_iv: "f8a903d09c2adf8939147eaafc2d5fa5"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.794694 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "e1251748d48f4fedb51b8d6c18320d56"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "6b8eb857eb550f17e74e5668ecf1b568"
server_key_iv: "f8a903d09c2adf8939147eaafc2d5fa5"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.799353 15379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:49:53.802589 15404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:53.803771 15379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:49:53.803907 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "e1251748d48f4fedb51b8d6c18320d56"
format_stamp: "Formatted at 2026-08-10 18:49:53 on dist-test-slave-6zbq"
server_key: "6b8eb857eb550f17e74e5668ecf1b568"
server_key_iv: "f8a903d09c2adf8939147eaafc2d5fa5"
server_key_version: "encryptionkey@0"
I20260810 18:49:53.804072 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:53.818998 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:53.822784 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:53.823029 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:53.823885 15379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:53.825086 15379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:53.825161 15379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:53.825275 15379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:53.825330 15379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:53.855016 15379 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:39135
I20260810 18:49:53.855059 15573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:39135 every 8 connection(s)
I20260810 18:49:53.856220 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:49:53.857288 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 15379
I20260810 18:49:53.857520 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:49:53.858330 30395 external_mini_cluster.cc:1428] Setting key 41a4927dc17f253dcd647c42c6db9f42
Aug 10 18:49:53 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387793, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:53.877229 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:53.858868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48161 (local address 127.29.174.254:39837)
0810 18:49:53.859311 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:53.859315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:53.860407 (+ 1092us) server_negotiation.cc:407] Connection header received
0810 18:49:53.862737 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:53.862741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:53.862809 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:53.862901 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:53.864679 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.865390 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.867393 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.867771 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.867867 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:53.871590 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:53.871596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:53.871601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:53.871641 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:53.874383 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:53.875063 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:53.875069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:53.875073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:53.875193 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:53.875604 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:53.875608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:53.875610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:53.875865 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:53.875999 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:53.876652 (+ 653us) server_negotiation.cc:299] Negotiation successful
0810 18:49:53.876874 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":224,"threads_started":1}
I20260810 18:49:53.878079 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:53.859411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39837 (local address 127.29.174.193:48161)
0810 18:49:53.860225 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:53.860293 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:53.862467 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:53.863146 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:53.863157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:53.863812 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:53.864522 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:53.864544 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.865547 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.865551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:53.867250 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:53.867276 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:53.867975 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:53.867982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:53.868242 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:53.868912 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:53.868938 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:53.871351 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:53.874517 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:53.874526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:53.874541 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:53.874938 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:53.875298 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:53.875303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:53.875306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:53.875487 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:53.875990 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:53.875998 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:53.876290 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 18:49:53.876510 (+ 220us) client_negotiation.cc:240] Negotiation successful
0810 18:49:53.877002 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":286,"threads_started":1}
I20260810 18:49:53.879552 15574 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39837
I20260810 18:49:53.879863 15574 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:53.880722 15574 heartbeater.cc:511] Master 127.29.174.254:39837 requested a full tablet report, sending...
I20260810 18:49:53.882997 15267 ts_manager.cc:194] Registered new tserver with Master: e1251748d48f4fedb51b8d6c18320d56 (127.29.174.193:39135)
I20260810 18:49:53.884892 15267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:48161
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:49:53.918020 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:54.232234 15586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:54.232482 15586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:54.232569 15586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:54.239446 15586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:54.239534 15586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:54.239646 15586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:49:54.248903 15586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:54.250043 15586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:54.251299 15586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:54.263465 15593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:54.263422 15598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:54.263662 15601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:54.264669 15586 server_base.cc:1034] running on GCE node
I20260810 18:49:54.265188 15586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:54.266000 15586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:54.267179 15586 hybrid_clock.cc:648] HybridClock initialized: now 1786387794267136 us; error 35 us; skew 500 ppm
I20260810 18:49:54.267431 15586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:54 dist-test-slave-6zbq krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:54.270236 15586 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:49:54.271488 15586 webserver.cc:460] Webserver started at http://127.29.174.194:43249/ using document root <none> and password file <none>
I20260810 18:49:54.272068 15586 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:54.272172 15586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:54.272446 15586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:54.275416 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "1cf88f9d3c904094ba32105b755bc52c"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "ecc1e9d06403e42625fc2a948bf307d6"
server_key_iv: "058a3eb34e28329a6e0a0521f528ef52"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.276365 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "1cf88f9d3c904094ba32105b755bc52c"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "ecc1e9d06403e42625fc2a948bf307d6"
server_key_iv: "058a3eb34e28329a6e0a0521f528ef52"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.282251 15586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:49:54.286079 15611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.287482 15586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:49:54.287639 15586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "1cf88f9d3c904094ba32105b755bc52c"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "ecc1e9d06403e42625fc2a948bf307d6"
server_key_iv: "058a3eb34e28329a6e0a0521f528ef52"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.287840 15586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:54.310148 15586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:54.313552 15586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:54.313788 15586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:54.314627 15586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:54.315871 15586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:54.315946 15586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.316032 15586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:54.316078 15586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.345459 15586 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:41259
I20260810 18:49:54.345523 15799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:41259 every 8 connection(s)
I20260810 18:49:54.346663 15586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:49:54.349843 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 15586
I20260810 18:49:54.350032 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:49:54.350517 30395 external_mini_cluster.cc:1428] Setting key c6ebc3fa4e29ce0c0fd600bea1d92dfc
Aug 10 18:49:54 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:54.365314 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.349028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:51801 (local address 127.29.174.254:39837)
0810 18:49:54.349229 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.349235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.350395 (+ 1160us) server_negotiation.cc:407] Connection header received
0810 18:49:54.352336 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.352342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.352423 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.352504 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:54.353888 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.354650 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.356101 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.356495 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.356603 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.359820 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:54.359825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:54.359828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:54.359858 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:54.362190 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.362870 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.362874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.362876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.363006 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.363408 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.363412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.363414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.363668 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:54.363818 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.364668 (+ 850us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.364911 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:49:54.366169 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.349442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39837 (local address 127.29.174.194:51801)
0810 18:49:54.350218 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.350268 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.352078 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.352737 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.352744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.353198 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:54.353722 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.353739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.354799 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.354803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.355961 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.355974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.356711 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.356719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.356922 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.357523 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:54.357545 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:54.359609 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:54.362316 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:54.362324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:54.362357 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:54.362759 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:54.363108 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:54.363113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:54.363117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:54.363293 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:54.363793 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:54.363800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:54.364101 (+ 301us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.364530 (+ 429us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.365058 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":320,"threads_started":1}
I20260810 18:49:54.367539 15800 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39837
I20260810 18:49:54.367805 15800 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:54.368472 15800 heartbeater.cc:511] Master 127.29.174.254:39837 requested a full tablet report, sending...
I20260810 18:49:54.369840 15267 ts_manager.cc:194] Registered new tserver with Master: 1cf88f9d3c904094ba32105b755bc52c (127.29.174.194:41259)
I20260810 18:49:54.370850 15267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:51801
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:49:54.406999 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:49:54.724864 15809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:49:54.725111 15809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:49:54.725208 15809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:49:54.732277 15809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:49:54.732368 15809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:49:54.732482 15809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:49:54.741521 15809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:49:54.742605 15809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:49:54.743774 15809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:49:54.755754 15826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:54.755897 15818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:49:54.755986 15822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:49:54.756501 15809 server_base.cc:1034] running on GCE node
I20260810 18:49:54.757009 15809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:49:54.757839 15809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:49:54.759027 15809 hybrid_clock.cc:648] HybridClock initialized: now 1786387794758977 us; error 43 us; skew 500 ppm
I20260810 18:49:54.759305 15809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:49:54 dist-test-slave-6zbq krb5kdc[15187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:49:54.762143 15809 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:49:54.763497 15809 webserver.cc:460] Webserver started at http://127.29.174.195:41745/ using document root <none> and password file <none>
I20260810 18:49:54.764101 15809 fs_manager.cc:359] Metadata directory not provided
I20260810 18:49:54.764215 15809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:49:54.764480 15809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:49:54.767221 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "d7635158823b41d48c532108975f4120"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "d6b1959673561f4f1ff5c4b9c59f6799"
server_key_iv: "3fc58bfa278841fedd5b75f6cfa13199"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.767947 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "d7635158823b41d48c532108975f4120"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "d6b1959673561f4f1ff5c4b9c59f6799"
server_key_iv: "3fc58bfa278841fedd5b75f6cfa13199"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.772575 15809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:49:54.775725 15840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.776885 15809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:49:54.777019 15809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "d7635158823b41d48c532108975f4120"
format_stamp: "Formatted at 2026-08-10 18:49:54 on dist-test-slave-6zbq"
server_key: "d6b1959673561f4f1ff5c4b9c59f6799"
server_key_iv: "3fc58bfa278841fedd5b75f6cfa13199"
server_key_version: "encryptionkey@0"
I20260810 18:49:54.777179 15809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:49:54.788182 15809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:49:54.791631 15809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:49:54.791901 15809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:49:54.792951 15809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:49:54.794387 15809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:49:54.794478 15809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.794600 15809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:49:54.794658 15809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:49:54.824352 15809 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:36143
I20260810 18:49:54.824395 16028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:36143 every 8 connection(s)
I20260810 18:49:54.825481 15809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:49:54.830766 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 15809
I20260810 18:49:54.830948 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:49:54.831447 30395 external_mini_cluster.cc:1428] Setting key fc9bbfbc597c356535dfee93efb54db3
Aug 10 18:49:54 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387794, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:49:54.843711 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.827880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:60079 (local address 127.29.174.254:39837)
0810 18:49:54.828095 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.828101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.829203 (+ 1102us) server_negotiation.cc:407] Connection header received
0810 18:49:54.830982 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.830988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.831076 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.831186 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:54.832723 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.833667 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.835364 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.835773 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.835868 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.838916 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:54.838921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:54.838924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:54.838956 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:54.841322 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.841924 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.841928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.841930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.842036 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.842382 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.842385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.842387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.842637 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:54.842744 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.843301 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.843459 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:49:54.844564 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.828344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39837 (local address 127.29.174.195:60079)
0810 18:49:54.829047 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.829092 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.830708 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.831365 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.831373 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.831884 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:49:54.832542 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.832568 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.833813 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.833817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.835241 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.835254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.835936 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.835940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.836119 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.836670 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:49:54.836691 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:49:54.838723 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:49:54.841444 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:54.841451 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:54.841460 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:54.841835 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:54.842108 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:49:54.842111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:49:54.842114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:49:54.842280 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:49:54.842729 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:49:54.842734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:49:54.843011 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.843218 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.843566 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":282,"threads_started":1}
I20260810 18:49:54.845791 16030 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39837
I20260810 18:49:54.846073 16030 heartbeater.cc:464] Registering TS with master...
I20260810 18:49:54.846827 16030 heartbeater.cc:511] Master 127.29.174.254:39837 requested a full tablet report, sending...
I20260810 18:49:54.848147 15266 ts_manager.cc:194] Registered new tserver with Master: d7635158823b41d48c532108975f4120 (127.29.174.195:36143)
I20260810 18:49:54.848944 15266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:60079
I20260810 18:49:54.858505 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:49:54.873346 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.862731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.29.174.254:39837)
0810 18:49:54.862864 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.862868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.863325 (+ 457us) server_negotiation.cc:407] Connection header received
0810 18:49:54.863426 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.863429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.863486 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.863557 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:54.864756 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.865423 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.866580 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.866992 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.867075 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.869068 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:54.869072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:54.869075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:54.869098 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:54.871212 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.871817 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.871820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.871823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.871941 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:54.872235 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:54.872238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:54.872240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:54.872513 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:54.872681 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.872921 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.873106 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:49:54.876770 15266 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:49:54.883285 15266 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:49:54.888291 15574 heartbeater.cc:503] Master 127.29.174.254:39837 was elected leader, sending a full tablet report...
I20260810 18:49:54.917558 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.895222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39135 (local address 127.0.0.1:34218)
0810 18:49:54.909124 (+ 13902us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.909147 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.909501 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.910483 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.910491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.910522 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:54.910891 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.910903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.912736 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.912741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.914693 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.914711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.916697 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.916704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.916894 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.916943 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.917055 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.917232 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13731,"thread_start_us":14052,"threads_started":1}
I20260810 18:49:54.917634 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.895824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.29.174.193:39135)
0810 18:49:54.902738 (+ 6914us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.902748 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.909248 (+ 6500us) server_negotiation.cc:407] Connection header received
0810 18:49:54.909691 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.909701 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.909961 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.910136 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:54.911099 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.912585 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.914848 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.916229 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.916404 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.916563 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.917145 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.917321 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6732,"thread_start_us":6581,"threads_started":1}
I20260810 18:49:54.918674 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.894723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:36143 (local address 127.0.0.1:56586)
0810 18:49:54.895398 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.895436 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.895597 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.911856 (+ 16259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.911860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.911878 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:54.912327 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.912340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.914251 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.914257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.916143 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.916158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.917896 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.917901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.918029 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.918060 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.918126 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.918245 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":161,"thread_start_us":285,"threads_started":1}
I20260810 18:49:54.919203 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.895752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41259 (local address 127.0.0.1:50308)
0810 18:49:54.909276 (+ 13524us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.909294 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.909422 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.911834 (+ 2412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.911838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.911862 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:54.912695 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.912709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.914503 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.914509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.916624 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.916636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.918416 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.918423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.918588 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.918625 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.918723 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.918841 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13393,"mutex_wait_us":32,"thread_start_us":13745,"threads_started":1}
I20260810 18:49:54.919685 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.894993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.29.174.195:36143)
0810 18:49:54.911455 (+ 16462us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.911460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.911483 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:49:54.911550 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.911555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.911694 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.912181 (+ 487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:54.912490 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.914093 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.916325 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.917663 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.919012 (+ 1349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.919169 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.919243 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.919390 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16336,"thread_start_us":16188,"threads_started":1}
I20260810 18:49:54.921526 15475 tablet_service.cc:1467] Processing CreateTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 (DEFAULT_TABLE table=test-table [id=e743f0790d8344f090e3da8d23d576d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:54.922719 15475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20065515b5ef4ef5af01b47f49b1ebb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:54.924299 15924 tablet_service.cc:1467] Processing CreateTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 (DEFAULT_TABLE table=test-table [id=e743f0790d8344f090e3da8d23d576d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:54.925179 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.896075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.29.174.194:41259)
0810 18:49:54.911292 (+ 15217us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.911301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.911328 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:49:54.911419 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.911426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.911636 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.911835 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:54.912920 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.914316 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.916799 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.918116 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.918322 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.918507 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.918802 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.924893 (+ 6091us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15056,"thread_start_us":15906,"threads_started":1}
I20260810 18:49:54.925536 15924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20065515b5ef4ef5af01b47f49b1ebb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:54.928684 15686 tablet_service.cc:1467] Processing CreateTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 (DEFAULT_TABLE table=test-table [id=e743f0790d8344f090e3da8d23d576d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:49:54.929977 15686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20065515b5ef4ef5af01b47f49b1ebb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:49:54.933902 16063 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Bootstrap starting.
I20260810 18:49:54.936699 16063 tablet_bootstrap.cc:654] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:54.937740 16063 log.cc:824] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:54.940047 16066 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Bootstrap starting.
I20260810 18:49:54.941128 16063 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: No bootstrap required, opened a new log
I20260810 18:49:54.941407 16063 ts_tablet_manager.cc:1397] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260810 18:49:54.943324 16067 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Bootstrap starting.
I20260810 18:49:54.943490 16066 tablet_bootstrap.cc:654] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:54.944687 16066 log.cc:824] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:54.946326 16067 tablet_bootstrap.cc:654] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Neither blocks nor log segments found. Creating new log.
I20260810 18:49:54.947427 16067 log.cc:824] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:49:54.948149 16066 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: No bootstrap required, opened a new log
I20260810 18:49:54.948400 16066 ts_tablet_manager.cc:1397] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:49:54.949667 16067 tablet_bootstrap.cc:492] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: No bootstrap required, opened a new log
I20260810 18:49:54.949893 16067 ts_tablet_manager.cc:1397] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 18:49:54.951287 16063 raft_consensus.cc:348] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.951699 16063 raft_consensus.cc:374] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:54.951793 16063 raft_consensus.cc:729] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1251748d48f4fedb51b8d6c18320d56, State: Initialized, Role: FOLLOWER
I20260810 18:49:54.952397 16063 consensus_queue.cc:260] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.954039 16063 ts_tablet_manager.cc:1428] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260810 18:49:54.958164 16066 raft_consensus.cc:348] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.958217 16067 raft_consensus.cc:348] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.958549 16066 raft_consensus.cc:374] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:54.958655 16067 raft_consensus.cc:374] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:49:54.958668 16066 raft_consensus.cc:729] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7635158823b41d48c532108975f4120, State: Initialized, Role: FOLLOWER
I20260810 18:49:54.958761 16067 raft_consensus.cc:729] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf88f9d3c904094ba32105b755bc52c, State: Initialized, Role: FOLLOWER
I20260810 18:49:54.959211 16066 consensus_queue.cc:260] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.959311 16067 consensus_queue.cc:260] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.960238 16030 heartbeater.cc:503] Master 127.29.174.254:39837 was elected leader, sending a full tablet report...
I20260810 18:49:54.960692 16066 ts_tablet_manager.cc:1428] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 18:49:54.961022 15800 heartbeater.cc:503] Master 127.29.174.254:39837 was elected leader, sending a full tablet report...
I20260810 18:49:54.961447 16067 ts_tablet_manager.cc:1428] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260810 18:49:54.977195 16073 raft_consensus.cc:482] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:49:54.977406 16073 raft_consensus.cc:504] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.978883 16073 leader_election.cc:288] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cf88f9d3c904094ba32105b755bc52c (127.29.174.194:41259), e1251748d48f4fedb51b8d6c18320d56 (127.29.174.193:39135)
I20260810 18:49:54.984809 16034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.979113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:39135 (local address 127.29.174.195:52489)
0810 18:49:54.979266 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.979285 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.979384 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.979778 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.979783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.979815 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:54.979988 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.979995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.981196 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.981200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.983042 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.983056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.984298 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.984302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.984434 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.984458 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.984535 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.984625 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 18:49:54.984930 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.979349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:52489 (local address 127.29.174.193:39135)
0810 18:49:54.979534 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.979539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.979552 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:49:54.979600 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.979604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.979655 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.979728 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:54.980117 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.981026 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.983204 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.984158 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.984315 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.984386 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.984576 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.984706 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:49:54.985837 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.979122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41259 (local address 127.29.174.195:45827)
0810 18:49:54.980046 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0810 18:49:54.980060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:49:54.980184 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:49:54.980463 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:49:54.980466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:49:54.980484 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:49:54.980739 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.980747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.981770 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.981775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.983563 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:49:54.983582 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.985175 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.985180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:49:54.985302 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.985332 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:49:54.985423 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:49:54.985543 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":282,"thread_start_us":288,"threads_started":1}
I20260810 18:49:54.985852 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:54.979247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:45827 (local address 127.29.174.194:41259)
0810 18:49:54.979424 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:54.979430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:54.980128 (+ 698us) server_negotiation.cc:407] Connection header received
0810 18:49:54.980300 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:54.980305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:54.980375 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:54.980458 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:49:54.980862 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.981636 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.983709 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:54.984807 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:54.984956 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:54.985025 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:54.985484 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 18:49:54.985618 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:49:54.985837 15512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20065515b5ef4ef5af01b47f49b1ebb3" candidate_uuid: "d7635158823b41d48c532108975f4120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1251748d48f4fedb51b8d6c18320d56" is_pre_election: true
I20260810 18:49:54.986392 15512 raft_consensus.cc:2454] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7635158823b41d48c532108975f4120 in term 0.
I20260810 18:49:54.987015 15846 leader_election.cc:302] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7635158823b41d48c532108975f4120, e1251748d48f4fedb51b8d6c18320d56; no voters:
I20260810 18:49:54.987396 16073 raft_consensus.cc:2781] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:49:54.987542 16073 raft_consensus.cc:482] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:49:54.987641 16073 raft_consensus.cc:3037] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:54.987905 15721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20065515b5ef4ef5af01b47f49b1ebb3" candidate_uuid: "d7635158823b41d48c532108975f4120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf88f9d3c904094ba32105b755bc52c" is_pre_election: true
I20260810 18:49:54.988291 15721 raft_consensus.cc:2454] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7635158823b41d48c532108975f4120 in term 0.
I20260810 18:49:54.989881 16073 raft_consensus.cc:504] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.990429 16073 leader_election.cc:288] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [CANDIDATE]: Term 1 election: Requested vote from peers 1cf88f9d3c904094ba32105b755bc52c (127.29.174.194:41259), e1251748d48f4fedb51b8d6c18320d56 (127.29.174.193:39135)
I20260810 18:49:54.990793 15721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20065515b5ef4ef5af01b47f49b1ebb3" candidate_uuid: "d7635158823b41d48c532108975f4120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cf88f9d3c904094ba32105b755bc52c"
I20260810 18:49:54.990824 15512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20065515b5ef4ef5af01b47f49b1ebb3" candidate_uuid: "d7635158823b41d48c532108975f4120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1251748d48f4fedb51b8d6c18320d56"
I20260810 18:49:54.990952 15721 raft_consensus.cc:3037] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:54.990973 15512 raft_consensus.cc:3037] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:49:54.993144 15512 raft_consensus.cc:2454] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7635158823b41d48c532108975f4120 in term 1.
I20260810 18:49:54.993184 15721 raft_consensus.cc:2454] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7635158823b41d48c532108975f4120 in term 1.
I20260810 18:49:54.993489 15846 leader_election.cc:302] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7635158823b41d48c532108975f4120, e1251748d48f4fedb51b8d6c18320d56; no voters:
I20260810 18:49:54.993762 16073 raft_consensus.cc:2781] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:49:54.994138 16073 raft_consensus.cc:686] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 1 LEADER]: Becoming Leader. State: Replica: d7635158823b41d48c532108975f4120, State: Running, Role: LEADER
I20260810 18:49:54.994578 16073 consensus_queue.cc:237] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } }
I20260810 18:49:54.998698 15267 catalog_manager.cc:5495] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7635158823b41d48c532108975f4120 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "d7635158823b41d48c532108975f4120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7635158823b41d48c532108975f4120" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 } health_report { overall_health: UNKNOWN } } }
I20260810 18:49:55.016999 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:55.012266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.29.174.195:36143)
0810 18:49:55.012446 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:55.012452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:55.012603 (+ 151us) server_negotiation.cc:407] Connection header received
0810 18:49:55.012749 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:55.012752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:55.012829 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:55.012940 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:49:55.013226 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.013901 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.015168 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.015548 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.015647 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:55.016079 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:49:55.016503 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:49:55.016603 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:55.016715 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 18:49:55.016781 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:49:55.024098 15512 raft_consensus.cc:1261] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 1 FOLLOWER]: Refusing update from remote peer d7635158823b41d48c532108975f4120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:55.024150 15721 raft_consensus.cc:1261] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 1 FOLLOWER]: Refusing update from remote peer d7635158823b41d48c532108975f4120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:49:55.024967 16078 consensus_queue.cc:1035] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cf88f9d3c904094ba32105b755bc52c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:55.025251 16073 consensus_queue.cc:1035] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1251748d48f4fedb51b8d6c18320d56" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 39135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:49:55.039896 16087 mvcc.cc:205] Tried to move back new op lower bound from 7317044408410173440 to 7317044408300990464. Current Snapshot: MvccSnapshot[applied={T|T < 7317044408410173440}]
I20260810 18:49:55.050803 15267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:55.051103 15267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:49:55.054473 15267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 453912337f7b4421a10131e0693428f0: Sending DeleteTablet for 3 replicas of tablet 20065515b5ef4ef5af01b47f49b1ebb3
I20260810 18:49:55.055418 15924 tablet_service.cc:1514] Processing DeleteTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:55 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:56586
I20260810 18:49:55.055505 15686 tablet_service.cc:1514] Processing DeleteTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:55 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:50308
I20260810 18:49:55.055985 15475 tablet_service.cc:1514] Processing DeleteTablet for tablet 20065515b5ef4ef5af01b47f49b1ebb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:49:55 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:34218
I20260810 18:49:55.057680 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 15379
I20260810 18:49:55.068040 16097 tablet_replica.cc:330] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: stopping tablet replica
I20260810 18:49:55.068539 16097 raft_consensus.cc:2229] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:49:55.069237 16097 raft_consensus.cc:2258] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:55.071245 16097 ts_tablet_manager.cc:1905] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:55.078792 16097 ts_tablet_manager.cc:1918] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:55.079041 16097 log.cc:1196] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/20065515b5ef4ef5af01b47f49b1ebb3
I20260810 18:49:55.079638 16097 ts_tablet_manager.cc:1939] T 20065515b5ef4ef5af01b47f49b1ebb3 P d7635158823b41d48c532108975f4120: Deleting consensus metadata
I20260810 18:49:55.081318 15242 catalog_manager.cc:4841] TS d7635158823b41d48c532108975f4120 (127.29.174.195:36143): tablet 20065515b5ef4ef5af01b47f49b1ebb3 (table test-table [id=e743f0790d8344f090e3da8d23d576d3]) successfully deleted
I20260810 18:49:55.083516 16100 tablet_replica.cc:330] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: stopping tablet replica
I20260810 18:49:55.083992 16100 raft_consensus.cc:2229] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:55.084666 16100 raft_consensus.cc:2258] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:55.085686 16098 tablet_replica.cc:330] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: stopping tablet replica
I20260810 18:49:55.086131 16098 raft_consensus.cc:2229] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:49:55.086694 16098 raft_consensus.cc:2258] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:49:55.087806 16100 ts_tablet_manager.cc:1905] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:49:55.088652 16098 ts_tablet_manager.cc:1905] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:49:55.094776 16100 ts_tablet_manager.cc:1918] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:55.095033 16100 log.cc:1196] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/20065515b5ef4ef5af01b47f49b1ebb3
I20260810 18:49:55.095602 16100 ts_tablet_manager.cc:1939] T 20065515b5ef4ef5af01b47f49b1ebb3 P e1251748d48f4fedb51b8d6c18320d56: Deleting consensus metadata
I20260810 18:49:55.095790 16098 ts_tablet_manager.cc:1918] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:49:55.095983 16098 log.cc:1196] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/20065515b5ef4ef5af01b47f49b1ebb3
I20260810 18:49:55.096800 16098 ts_tablet_manager.cc:1939] T 20065515b5ef4ef5af01b47f49b1ebb3 P 1cf88f9d3c904094ba32105b755bc52c: Deleting consensus metadata
I20260810 18:49:55.097468 15246 catalog_manager.cc:4841] TS e1251748d48f4fedb51b8d6c18320d56 (127.29.174.193:39135): tablet 20065515b5ef4ef5af01b47f49b1ebb3 (table test-table [id=e743f0790d8344f090e3da8d23d576d3]) successfully deleted
I20260810 18:49:55.098453 15242 catalog_manager.cc:4841] TS 1cf88f9d3c904094ba32105b755bc52c (127.29.174.194:41259): tablet 20065515b5ef4ef5af01b47f49b1ebb3 (table test-table [id=e743f0790d8344f090e3da8d23d576d3]) successfully deleted
I20260810 18:49:55.100723 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:55.059923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.29.174.193:39135)
0810 18:49:55.060098 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:55.060105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:55.082737 (+ 22632us) server_negotiation.cc:407] Connection header received
0810 18:49:55.082914 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:55.082920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:55.083015 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:55.083126 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:55.084925 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.086314 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.088205 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.088888 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.088990 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:55.094256 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:55.094271 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:55.094288 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:55.094327 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:55.098098 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.098957 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.098964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.098966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.099088 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.099462 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.099465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.099467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.099814 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:55.100029 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:55.100332 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:49:55.100484 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:49:55.101375 15569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:55.186391 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 15379
I20260810 18:49:55.211625 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 15586
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:49:55.226505 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:55.212423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.29.174.194:41259)
0810 18:49:55.212617 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:55.212623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:55.212643 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:49:55.212735 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:55.212741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:55.212824 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:55.212908 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:55.214104 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.214938 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.216195 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.216656 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.216765 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:55.221182 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:55.221197 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:55.221212 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:55.221243 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:55.223905 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.224632 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.224638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.224640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.224753 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.225086 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.225089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.225091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.225445 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:55.225648 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:55.225956 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 18:49:55.226108 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:49:55.227195 15793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:55.313648 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 15586
I20260810 18:49:55.337956 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 15809
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[15187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:49:55.353422 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:49:55.338711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.29.174.195:36143)
0810 18:49:55.338904 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:49:55.338910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:49:55.338930 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:49:55.339067 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:49:55.339072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:49:55.339154 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:49:55.339261 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:49:55.340469 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.341305 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.342568 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:49:55.343046 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:49:55.343161 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:49:55.347620 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:49:55.347635 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:49:55.347650 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:49:55.347681 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:49:55.350523 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.351109 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.351116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.351118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.351305 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:49:55.351765 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:49:55.351769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:49:55.351771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:49:55.352140 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:49:55.352375 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 18:49:55.352845 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 18:49:55.353015 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:49:55.354182 16022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:55.471755 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 15809
I20260810 18:49:55.494204 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 15211
I20260810 18:49:55.495306 15322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:49:55.599565 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 15211
2026-08-10T18:49:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4185 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[16114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[16114](info): set up 2 sockets
Aug 10 18:49:55 dist-test-slave-6zbq krb5kdc[16114](info): commencing operation
krb5kdc: starting...
W20260810 18:49:56.828749 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:49:56 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:49:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangeradmin_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangeradmin_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangerlookup_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangerlookup_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
I20260810 18:49:57.142261 30395 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY4kHxe/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/postgres -l logfile start
2026-08-10 18:49:59.410 UTC [16170] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:49:59.410 UTC [16170] LOG: listening on IPv4 address "127.29.174.212", port 42137
2026-08-10 18:49:59.413 UTC [16170] LOG: listening on Unix socket "/tmp/.s.PGSQL.42137"
2026-08-10 18:49:59.434 UTC [16176] LOG: database system was shut down at 2026-08-10 18:49:58 UTC
2026-08-10 18:49:59.438 UTC [16170] LOG: database system is ready to accept connections
I20260810 18:50:00.553361 30395 mini_postgres.cc:96] Postgres bound to 42137
2026-08-10 18:50:00.578 UTC [16185] FATAL: database "slave" does not exist
127.29.174.212:42137 - accepting connections
I20260810 18:50:00.579572 30395 mini_ranger.cc:161] Starting Ranger...
I20260810 18:50:00.611961 30395 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:50:01.046118 30395 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:50:01.046228 30395 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin
2026-08-10 18:50:01,379 [I] DB FLAVOR :POSTGRES
2026-08-10 18:50:01,380 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:50:01,380 [I] Checking connection..
2026-08-10 18:50:01,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:50:01,642 [I] Checking connection passed.
2026-08-10 18:50:01,643 [I] --------- Verifying version history table ---------
2026-08-10 18:50:01,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:01,881 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:01,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:02,139 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:02,139 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:50:02,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:50:02.391 UTC [16296] WARNING: there is no transaction in progress
2026-08-10 18:50:02,397 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:50:02,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:02,626 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:50:02,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:50:02,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:50:02,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:50:03,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-10 18:50:03,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:03,628 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:50:03,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:03,858 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:50:03,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:04,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:04,358 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:50:04,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:50:05.258 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.273 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.287 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.301 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.314 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.330 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.379 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.384 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.397 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.405 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.414 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.426 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.434 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.440 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.446 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05.451 UTC [16480] WARNING: there is no transaction in progress
2026-08-10 18:50:05,993 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:50:05,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:50:06,234 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:50:06,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:06,474 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:50:06,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:06,723 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:50:06,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:06,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:07,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-10 18:50:07,441 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:50:07,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:50:07,674 [I] DB_PATCHES have already been applied
I20260810 18:50:07.679874 30395 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY4kHxe/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:50:07.679955 30395 mini_ranger.cc:193] Using host: 127.29.174.212
I20260810 18:50:07.700624 30395 mini_ranger.cc:227] Ranger admin URL: http://127.29.174.212:48659
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:50:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:50:09.007093 30395 mini_ranger.cc:160] Time spent starting Ranger: real 8.428s user 0.004s sys 0.056s
Aug 10, 2026 6:50:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:50:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:50:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:50:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:50:28 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:50:29.475276 30395 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangerkms_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/rangerkms_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.174.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.174.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:50:29 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387829, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:50:29.647622 30395 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:50:29.675101 30395 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:50:30.405927 30395 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:50:30.406092 30395 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms
I20260810 18:50:30.406211 30395 mini_ranger_kms.cc:227] Using postgres at 127.29.174.212:42137
2026-08-10 18:50:31,076 [I] DB FLAVOR :POSTGRES
2026-08-10 18:50:31,076 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:50:31,076 [I] Checking connection
2026-08-10 18:50:31,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:50:31,308 [I] connection success
2026-08-10 18:50:31,308 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:50:31,308 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:50:31,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:50:31,529 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:50:31,530 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:50:31,530 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:50:31,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:42137/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:50:31,796 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:50:32.283305 30395 mini_ranger_kms.cc:310] Created kms service
I20260810 18:50:32.462234 30395 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:50:32.504128 30395 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:50:32 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387829, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:32.916699 30395 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:50:32.918326 30395 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:50:32.918434 30395 mini_ranger_kms.cc:240] Using host: 127.29.174.212
I20260810 18:50:32.942579 30395 mini_ranger_kms.cc:276] Ranger KMS PID: 16798
I20260810 18:50:32.942744 30395 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.174.212:35079
Aug 10, 2026 6:50:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:50:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:50:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:50:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:50:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:50:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:50:35 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387835, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:35.520467 30395 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.873s user 0.134s sys 0.412s
I20260810 18:50:35.520851 30395 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:50:35.521091 30395 mini_ranger_kms.cc:401] 127.29.174.212:35079/kms/v1/keys
Aug 10 18:50:35 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387835, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
Aug 10 18:50:36 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387835, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
Aug 10 18:50:38 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:50:38.453954 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41185
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:41185
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:35079/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:50:39.021539 16875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:39.021786 16875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:39.021879 16875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:39.028759 16875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:50:39.028849 16875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:39.028900 16875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:39.028954 16875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:50:39.029001 16875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:50:39.038771 16875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.174.212:35079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:41185
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:50:39.039815 16875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:39.040994 16875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:39.052783 16887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:39.052846 16888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:39.052816 16891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:39.053725 16875 server_base.cc:1034] running on GCE node
I20260810 18:50:39.054431 16875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:39.055634 16875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:39.056828 16875 hybrid_clock.cc:648] HybridClock initialized: now 1786387839056780 us; error 41 us; skew 500 ppm
I20260810 18:50:39.057148 16875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:50:39 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387839, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:39.062886 16875 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:50:39.064190 16875 webserver.cc:460] Webserver started at http://127.29.174.254:36897/ using document root <none> and password file <none>
I20260810 18:50:39.064716 16875 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:39.064817 16875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:39.065045 16875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:39 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387839, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:39.118162 16875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "64e02a0fa80044c7a0006dc212eb2df9"
format_stamp: "Formatted at 2026-08-10 18:50:39 on dist-test-slave-6zbq"
server_key: "02fd60bc41d66fc2b16f6f5a12ae5431"
server_key_iv: "dac2f9858cf104a86fa4d30974c0d36b"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:39.119035 16875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "64e02a0fa80044c7a0006dc212eb2df9"
format_stamp: "Formatted at 2026-08-10 18:50:39 on dist-test-slave-6zbq"
server_key: "02fd60bc41d66fc2b16f6f5a12ae5431"
server_key_iv: "dac2f9858cf104a86fa4d30974c0d36b"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:39.124221 16875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:50:39.140323 16900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:39.142674 16875 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:50:39.142838 16875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "64e02a0fa80044c7a0006dc212eb2df9"
format_stamp: "Formatted at 2026-08-10 18:50:39 on dist-test-slave-6zbq"
server_key: "02fd60bc41d66fc2b16f6f5a12ae5431"
server_key_iv: "dac2f9858cf104a86fa4d30974c0d36b"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:39.143035 16875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:39.164495 16875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:39.168313 16875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:39.168594 16875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:39.187274 16875 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:41185
I20260810 18:50:39.187294 16998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:41185 every 8 connection(s)
I20260810 18:50:39.188565 16875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:50:39.192221 16999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:39.192359 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 16875
I20260810 18:50:39.192555 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:50:39.193342 30395 external_mini_cluster.cc:1428] Setting key 28d74a966bfc45e89b45457038847e1b
I20260810 18:50:39.207221 16999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: Bootstrap starting.
Aug 10 18:50:39 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:50:39.210566 16999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:39.211638 16999 log.cc:824] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:39.214515 16999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: No bootstrap required, opened a new log
I20260810 18:50:39.219354 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:39.195328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.29.174.254:41185)
0810 18:50:39.196033 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:39.196049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:39.196105 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:50:39.197919 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:39.197946 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:39.198457 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:39.198884 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:39.200422 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:39.202309 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:39.204118 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:39.205485 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:39.205832 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:39.210487 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:39.210499 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:39.210518 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:39.210568 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:39.214526 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:39.215398 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:39.215406 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:39.215414 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:39.215624 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:39.216101 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:39.216107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:39.216110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:39.216802 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:39.217101 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:39.217560 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 18:50:39.217956 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":307,"threads_started":1}
I20260810 18:50:39.223169 16999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } }
I20260810 18:50:39.223489 16999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:39.223572 16999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e02a0fa80044c7a0006dc212eb2df9, State: Initialized, Role: FOLLOWER
I20260810 18:50:39.224094 16999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } }
I20260810 18:50:39.224326 16999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:50:39.224447 16999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:50:39.224586 16999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:39.226995 16999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } }
I20260810 18:50:39.227427 16999 leader_election.cc:302] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e02a0fa80044c7a0006dc212eb2df9; no voters:
I20260810 18:50:39.228055 16999 leader_election.cc:288] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:50:39.228161 17006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:39.230135 16999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:50:39.230139 17006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [term 1 LEADER]: Becoming Leader. State: Replica: 64e02a0fa80044c7a0006dc212eb2df9, State: Running, Role: LEADER
I20260810 18:50:39.230731 17006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } }
I20260810 18:50:39.234746 17012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e02a0fa80044c7a0006dc212eb2df9. Latest consensus state: current_term: 1 leader_uuid: "64e02a0fa80044c7a0006dc212eb2df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } } }
I20260810 18:50:39.235085 17012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:39.235522 17007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e02a0fa80044c7a0006dc212eb2df9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e02a0fa80044c7a0006dc212eb2df9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41185 } } }
I20260810 18:50:39.235833 17007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:50:39.236331 17035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:50:39.241714 17035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:50:39.243461 16999 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:50:39.251487 17035 catalog_manager.cc:1347] Generated new cluster ID: e470aa96a69f4b6ca04f06265969b722
I20260810 18:50:39.251606 17035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:50:39.276367 17035 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:50:39.277813 17035 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:50:39.286832 17035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: Generated new TSK 0
I20260810 18:50:39.287568 17035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:50:40.106118 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:35079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:40 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:50:40.630020 17075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:40.630479 17075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:40.630728 17075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:40.644152 17075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:40.644340 17075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:40.644556 17075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:50:40.662823 17075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.174.212:35079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:50:40.664860 17075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:40.666271 17075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:40.687749 17089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:40.692713 17092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:40.693279 17090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:40.695628 17075 server_base.cc:1034] running on GCE node
I20260810 18:50:40.696168 17075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:40.697053 17075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:40.698254 17075 hybrid_clock.cc:648] HybridClock initialized: now 1786387840698197 us; error 53 us; skew 500 ppm
I20260810 18:50:40.698566 17075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:50:40 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:40.702579 17075 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:50:40.705541 17075 webserver.cc:460] Webserver started at http://127.29.174.193:41681/ using document root <none> and password file <none>
I20260810 18:50:40.706398 17075 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:40.706602 17075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:40.707037 17075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:40 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:40.726122 17075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "bf67fb1d74984c2ca6207e8dfa78678a"
format_stamp: "Formatted at 2026-08-10 18:50:40 on dist-test-slave-6zbq"
server_key: "3f74b05d3f7cbf6e08a71172e5b47c53"
server_key_iv: "6867a37cf0ca950e7006e36d38b7eae4"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:40.727489 17075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "bf67fb1d74984c2ca6207e8dfa78678a"
format_stamp: "Formatted at 2026-08-10 18:50:40 on dist-test-slave-6zbq"
server_key: "3f74b05d3f7cbf6e08a71172e5b47c53"
server_key_iv: "6867a37cf0ca950e7006e36d38b7eae4"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:40.745471 17075 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.003s
I20260810 18:50:40.769704 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:40.771522 17075 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:50:40.771756 17075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "bf67fb1d74984c2ca6207e8dfa78678a"
format_stamp: "Formatted at 2026-08-10 18:50:40 on dist-test-slave-6zbq"
server_key: "3f74b05d3f7cbf6e08a71172e5b47c53"
server_key_iv: "6867a37cf0ca950e7006e36d38b7eae4"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:40.772081 17075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:40.802181 17075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:40.810045 17075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:40.810557 17075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:40.812017 17075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:40.814254 17075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:40.814381 17075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:40.814517 17075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:40.814625 17075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:50:40 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:41.063832 17075 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:46035
I20260810 18:50:41.063835 17288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:46035 every 8 connection(s)
I20260810 18:50:41.065320 17075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:50:41.067764 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 17075
I20260810 18:50:41.067997 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:50:41.069027 30395 external_mini_cluster.cc:1428] Setting key 155e9a7715569544228d3b58cf9e5679
Aug 10 18:50:41 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:50:41.110248 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:41.077504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:43539 (local address 127.29.174.254:41185)
0810 18:50:41.088912 (+ 11408us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:41.088919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:41.088939 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:50:41.088999 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:41.089004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:41.089081 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:41.089204 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:41.091504 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.092908 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.095607 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.096447 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.096538 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:41.101488 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:41.101495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:41.101499 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:41.101536 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:41.105783 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:41.106876 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:41.106882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:41.106887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:41.107041 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:41.107619 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:41.107624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:41.107627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:41.108022 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:41.108270 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:41.109400 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0810 18:50:41.109633 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11268,"thread_start_us":11064,"threads_started":1}
I20260810 18:50:41.113016 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:41.072595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41185 (local address 127.29.174.193:43539)
0810 18:50:41.083281 (+ 10686us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:41.083336 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:41.086168 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:41.089772 (+ 3604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:41.089785 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:41.090561 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:41.091323 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:41.091344 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.093155 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.093160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:41.095438 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:41.095453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.096964 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.096972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:41.097337 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:41.098220 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:41.098249 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:41.101176 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:41.105984 (+ 4808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:41.105994 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:41.106009 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:41.106728 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:41.107184 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:41.107189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:41.107193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:41.107470 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:41.108367 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:41.108375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:41.108876 (+ 501us) client_negotiation.cc:769] Sending connection context
0810 18:50:41.110823 (+ 1947us) client_negotiation.cc:240] Negotiation successful
0810 18:50:41.111440 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10355,"thread_start_us":10127,"threads_started":1}
I20260810 18:50:41.115345 17289 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41185
I20260810 18:50:41.115669 17289 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:41.116568 17289 heartbeater.cc:511] Master 127.29.174.254:41185 requested a full tablet report, sending...
I20260810 18:50:41.118999 16933 ts_manager.cc:194] Registered new tserver with Master: bf67fb1d74984c2ca6207e8dfa78678a (127.29.174.193:46035)
I20260810 18:50:41.121241 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:43539
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:50:41.166129 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:35079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:50:41 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387840, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
W20260810 18:50:41.753677 17297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:41.753983 17297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:41.754174 17297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:41.762524 17297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:41.762633 17297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:41.762792 17297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:50:41.773394 17297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.174.212:35079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:50:41.774595 17297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:41.775840 17297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:41.790213 17321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:41.790654 17322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:41.790982 17324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:41.792032 17297 server_base.cc:1034] running on GCE node
I20260810 18:50:41.792598 17297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:41.793622 17297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:41.794819 17297 hybrid_clock.cc:648] HybridClock initialized: now 1786387841794768 us; error 35 us; skew 500 ppm
I20260810 18:50:41.795089 17297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:50:41 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387841, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:41.798784 17297 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:50:41.800228 17297 webserver.cc:460] Webserver started at http://127.29.174.194:41087/ using document root <none> and password file <none>
I20260810 18:50:41.800899 17297 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:41.801086 17297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:41.801481 17297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:41 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387841, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:41.815637 17297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "83e11ae057164ab1b0c7d549cdadba72"
format_stamp: "Formatted at 2026-08-10 18:50:41 on dist-test-slave-6zbq"
server_key: "49c19d3acd05bde623bc7ccf64409700"
server_key_iv: "ad45ba0167f5e006971c95373d39984f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:41.816622 17297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "83e11ae057164ab1b0c7d549cdadba72"
format_stamp: "Formatted at 2026-08-10 18:50:41 on dist-test-slave-6zbq"
server_key: "49c19d3acd05bde623bc7ccf64409700"
server_key_iv: "ad45ba0167f5e006971c95373d39984f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:41.822618 17297 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.006s
I20260810 18:50:41.841264 17339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:41.842998 17297 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:50:41.843209 17297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "83e11ae057164ab1b0c7d549cdadba72"
format_stamp: "Formatted at 2026-08-10 18:50:41 on dist-test-slave-6zbq"
server_key: "49c19d3acd05bde623bc7ccf64409700"
server_key_iv: "ad45ba0167f5e006971c95373d39984f"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:41.843458 17297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:41.869817 17297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:41.874136 17297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:41.874641 17297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:41.875958 17297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:41.877411 17297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:41.877511 17297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:41.877641 17297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:41.877687 17297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:41.934029 17297 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:34073
I20260810 18:50:41.934094 17543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:34073 every 8 connection(s)
I20260810 18:50:41.935696 17297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:50:41.945544 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 17297
I20260810 18:50:41.945699 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:50:41.947022 30395 external_mini_cluster.cc:1428] Setting key 63ebb710e72f97cc099656e54e6abd2a
Aug 10 18:50:41 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387841, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:50:41.959646 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:41.938313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:35759 (local address 127.29.174.254:41185)
0810 18:50:41.939067 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:41.939072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:41.940132 (+ 1060us) server_negotiation.cc:407] Connection header received
0810 18:50:41.941990 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:41.941994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:41.942074 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:41.942167 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:41.943951 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.944694 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.948252 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.948875 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.949005 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:41.952946 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:41.952952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:41.952955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:41.953007 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:41.956285 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:41.957222 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:41.957228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:41.957232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:41.957376 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:41.957820 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:41.957825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:41.957829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:41.958156 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:41.958301 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:41.959134 (+ 833us) server_negotiation.cc:299] Negotiation successful
0810 18:50:41.959300 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":628,"thread_start_us":456,"threads_started":1}
I20260810 18:50:41.960867 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:41.938918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41185 (local address 127.29.174.194:35759)
0810 18:50:41.939984 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:41.940022 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:41.941754 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:41.942440 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:41.942449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:41.943072 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:41.943801 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:41.943827 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.944867 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.944871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:41.948086 (+ 3215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:41.948101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:41.949228 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:41.949238 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:41.949628 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:41.950436 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:41.950459 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:41.952706 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:41.956479 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:41.956488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:41.956503 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:41.957107 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:41.957479 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:41.957483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:41.957486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:41.957718 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:41.958272 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:41.958279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:41.958689 (+ 410us) client_negotiation.cc:769] Sending connection context
0810 18:50:41.958962 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 18:50:41.959527 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"thread_start_us":509,"threads_started":1}
I20260810 18:50:41.962288 17545 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41185
I20260810 18:50:41.962594 17545 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:41.963449 17545 heartbeater.cc:511] Master 127.29.174.254:41185 requested a full tablet report, sending...
I20260810 18:50:41.964896 16933 ts_manager.cc:194] Registered new tserver with Master: 83e11ae057164ab1b0c7d549cdadba72 (127.29.174.194:34073)
I20260810 18:50:41.965669 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:35759
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:50:42.003574 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:35079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:50:42.124715 17289 heartbeater.cc:503] Master 127.29.174.254:41185 was elected leader, sending a full tablet report...
W20260810 18:50:42.481098 17554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:50:42.481419 17554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:50:42.481644 17554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:50:42.490401 17554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:50:42.490548 17554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:50:42.490756 17554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:50:42.500792 17554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.174.212:35079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:50:42.502048 17554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:50:42.503381 17554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:50:42.518127 17562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:42.518328 17563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:50:42.518846 17565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:50:42.524163 17554 server_base.cc:1034] running on GCE node
I20260810 18:50:42.524793 17554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:50:42.525840 17554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:50:42.527076 17554 hybrid_clock.cc:648] HybridClock initialized: now 1786387842526982 us; error 33 us; skew 500 ppm
I20260810 18:50:42.527420 17554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:50:42 dist-test-slave-6zbq krb5kdc[16114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:50:42.531834 17554 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:50:42.533684 17554 webserver.cc:460] Webserver started at http://127.29.174.195:43807/ using document root <none> and password file <none>
I20260810 18:50:42.534549 17554 fs_manager.cc:359] Metadata directory not provided
I20260810 18:50:42.534739 17554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:50:42.535135 17554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:50:42 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:50:42.551656 17554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "97087064ffcc4fd5b22837545712dc76"
format_stamp: "Formatted at 2026-08-10 18:50:42 on dist-test-slave-6zbq"
server_key: "a6dda4e8f99915a401c9cae40a5092bb"
server_key_iv: "1dfe10a5d3a086f46733d8c899032ce2"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:42.552652 17554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "97087064ffcc4fd5b22837545712dc76"
format_stamp: "Formatted at 2026-08-10 18:50:42 on dist-test-slave-6zbq"
server_key: "a6dda4e8f99915a401c9cae40a5092bb"
server_key_iv: "1dfe10a5d3a086f46733d8c899032ce2"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:42.558921 17554 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260810 18:50:42.576056 17578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:42.577561 17554 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 18:50:42.577783 17554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "97087064ffcc4fd5b22837545712dc76"
format_stamp: "Formatted at 2026-08-10 18:50:42 on dist-test-slave-6zbq"
server_key: "a6dda4e8f99915a401c9cae40a5092bb"
server_key_iv: "1dfe10a5d3a086f46733d8c899032ce2"
server_key_version: "kuduclusterkey@0"
I20260810 18:50:42.578053 17554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:50:42.593183 17554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:50:42.598148 17554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:50:42.598676 17554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:50:42.599821 17554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:50:42.601706 17554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:50:42.601836 17554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:42.602025 17554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:50:42.602133 17554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:50:42.655203 17554 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:45091
I20260810 18:50:42.655195 17776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:45091 every 8 connection(s)
I20260810 18:50:42.656627 17554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:50:42.657094 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 17554
I20260810 18:50:42.657261 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:50:42.657732 30395 external_mini_cluster.cc:1428] Setting key 8cf78ec2d3b33f8e2be3e0ce207ab891
Aug 10 18:50:42 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387842, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:50:42.678776 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.659717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:46689 (local address 127.29.174.254:41185)
0810 18:50:42.660718 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.660725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.661972 (+ 1247us) server_negotiation.cc:407] Connection header received
0810 18:50:42.664513 (+ 2541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.664517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.664594 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.664686 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:42.666113 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.666884 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.668377 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.669186 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.669280 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.673179 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:42.673186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:42.673191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:42.673231 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:42.676273 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:42.676874 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:42.676877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:42.676879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:42.676993 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:42.677367 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:42.677370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:42.677372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:42.677636 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:42.677744 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.678328 (+ 584us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.678527 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":2101,"threads_started":1}
I20260810 18:50:42.679811 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.660433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41185 (local address 127.29.174.195:46689)
0810 18:50:42.661826 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.661871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.664280 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.664883 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.664891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.665420 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:50:42.665965 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.665982 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.667007 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.667011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.668251 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.668264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.669357 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.669362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.669551 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.670233 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:50:42.670254 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:50:42.672936 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:50:42.676394 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:42.676400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:42.676409 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:42.676790 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:42.677091 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:50:42.677095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:50:42.677112 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:50:42.677284 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:50:42.677723 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:50:42.677728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:50:42.678005 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.678242 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.678684 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1021,"thread_start_us":669,"threads_started":1}
I20260810 18:50:42.681289 17777 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41185
I20260810 18:50:42.681622 17777 heartbeater.cc:464] Registering TS with master...
I20260810 18:50:42.682618 17777 heartbeater.cc:511] Master 127.29.174.254:41185 requested a full tablet report, sending...
I20260810 18:50:42.684001 16933 ts_manager.cc:194] Registered new tserver with Master: 97087064ffcc4fd5b22837545712dc76 (127.29.174.195:45091)
I20260810 18:50:42.685216 16933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:46689
I20260810 18:50:42.686399 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:50:42.705366 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.691836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.29.174.254:41185)
0810 18:50:42.692009 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.692016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.692530 (+ 514us) server_negotiation.cc:407] Connection header received
0810 18:50:42.692693 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.692697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.692799 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.692872 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:42.694883 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.695789 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.697271 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.697850 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.697939 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.700456 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:42.700462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:42.700465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:42.700493 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:42.703131 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:42.703717 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:42.703721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:42.703723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:42.703868 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:42.704156 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:42.704159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:42.704161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:42.704390 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:42.704600 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.704798 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.704998 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:50:42.709390 16932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:50:42.718331 16932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:50:42.762954 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.731733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46035 (local address 127.0.0.1:38932)
0810 18:50:42.744912 (+ 13179us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.744956 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.745207 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.753651 (+ 8444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.753661 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.753698 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:42.754844 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.754856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756897 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.756901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.759833 (+ 2932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.759846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.762040 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.762048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.762236 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.762290 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.762426 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.762560 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12863,"mutex_wait_us":138,"spinlock_wait_cycles":1792,"thread_start_us":15805,"threads_started":1}
I20260810 18:50:42.763350 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.732269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38932 (local address 127.29.174.193:46035)
0810 18:50:42.753234 (+ 20965us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.753240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.753262 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:50:42.753338 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.753343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.753476 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.753648 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:42.755035 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756636 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.760014 (+ 3378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.761712 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.762003 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.762184 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.762529 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.762732 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20816,"thread_start_us":20634,"threads_started":1}
I20260810 18:50:42.763880 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.732175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.29.174.194:34073)
0810 18:50:42.751868 (+ 19693us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.751877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.751907 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:50:42.752019 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.752028 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.752314 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.752522 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:42.754474 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756388 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.760270 (+ 3882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.761782 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.762001 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.762184 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.763352 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.763528 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19505,"thread_start_us":20367,"threads_started":1}
I20260810 18:50:42.764624 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.731835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:34073 (local address 127.0.0.1:33896)
0810 18:50:42.749842 (+ 18007us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.749863 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.750016 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.753233 (+ 3217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.753238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.753265 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:42.754208 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.754224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756566 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.756571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.760128 (+ 3557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.760138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.762442 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.762451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.762631 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.762671 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.762751 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.764154 (+ 1403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15891,"mutex_wait_us":137,"spinlock_wait_cycles":2816,"thread_start_us":12494,"threads_started":1}
I20260810 18:50:42.765712 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.733918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:45091 (local address 127.0.0.1:49722)
0810 18:50:42.751562 (+ 17644us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.751588 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.751967 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.752948 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.752956 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.752987 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:42.754433 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.754450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756566 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.756583 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.760104 (+ 3521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.760123 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.764774 (+ 4651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.764784 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.765010 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.765035 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.765108 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.765228 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17534,"mutex_wait_us":8,"thread_start_us":17797,"threads_started":1}
I20260810 18:50:42.765972 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.734247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.29.174.195:45091)
0810 18:50:42.751227 (+ 16980us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.751237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.751689 (+ 452us) server_negotiation.cc:407] Connection header received
0810 18:50:42.752207 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.752215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.752445 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.752697 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:42.754690 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.756410 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.760283 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.761805 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.762018 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.762200 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.765235 (+ 3035us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.765409 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16773,"thread_start_us":16990,"threads_started":1}
I20260810 18:50:42.767674 17191 tablet_service.cc:1467] Processing CreateTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 (DEFAULT_TABLE table=test-table [id=52faa23b0c14431e8798a334ff2cc08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:42.768945 17191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68722bf0ccc043528e2cd6af0bd14530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:42.772328 17679 tablet_service.cc:1467] Processing CreateTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 (DEFAULT_TABLE table=test-table [id=52faa23b0c14431e8798a334ff2cc08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:42.773921 17679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68722bf0ccc043528e2cd6af0bd14530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:42.776674 17434 tablet_service.cc:1467] Processing CreateTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 (DEFAULT_TABLE table=test-table [id=52faa23b0c14431e8798a334ff2cc08c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:50:42.778201 17434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68722bf0ccc043528e2cd6af0bd14530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:50:42.785356 17815 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Bootstrap starting.
I20260810 18:50:42.788851 17816 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Bootstrap starting.
I20260810 18:50:42.790504 17815 tablet_bootstrap.cc:654] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:42.791975 17815 log.cc:824] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:42.793612 17816 tablet_bootstrap.cc:654] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:42.795025 17818 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Bootstrap starting.
I20260810 18:50:42.795295 17816 log.cc:824] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:42.796890 17815 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: No bootstrap required, opened a new log
I20260810 18:50:42.797248 17815 ts_tablet_manager.cc:1397] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260810 18:50:42.799414 17818 tablet_bootstrap.cc:654] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Neither blocks nor log segments found. Creating new log.
I20260810 18:50:42.799537 17816 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: No bootstrap required, opened a new log
I20260810 18:50:42.799892 17816 ts_tablet_manager.cc:1397] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260810 18:50:42.800927 17818 log.cc:824] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Log is configured to *not* fsync() on all Append() calls
I20260810 18:50:42.804065 17818 tablet_bootstrap.cc:492] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: No bootstrap required, opened a new log
I20260810 18:50:42.804320 17818 ts_tablet_manager.cc:1397] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 18:50:42.807267 17815 raft_consensus.cc:348] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.807753 17815 raft_consensus.cc:374] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:42.807910 17815 raft_consensus.cc:729] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf67fb1d74984c2ca6207e8dfa78678a, State: Initialized, Role: FOLLOWER
I20260810 18:50:42.808534 17815 consensus_queue.cc:260] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.810287 17815 ts_tablet_manager.cc:1428] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 18:50:42.812943 17816 raft_consensus.cc:348] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.813411 17816 raft_consensus.cc:374] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:42.813575 17816 raft_consensus.cc:729] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97087064ffcc4fd5b22837545712dc76, State: Initialized, Role: FOLLOWER
I20260810 18:50:42.814236 17816 consensus_queue.cc:260] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.815541 17777 heartbeater.cc:503] Master 127.29.174.254:41185 was elected leader, sending a full tablet report...
I20260810 18:50:42.815902 17818 raft_consensus.cc:348] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.816301 17816 ts_tablet_manager.cc:1428] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Time spent starting tablet: real 0.016s user 0.011s sys 0.006s
I20260810 18:50:42.816485 17818 raft_consensus.cc:374] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:50:42.816645 17818 raft_consensus.cc:729] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83e11ae057164ab1b0c7d549cdadba72, State: Initialized, Role: FOLLOWER
I20260810 18:50:42.817317 17818 consensus_queue.cc:260] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.818749 17545 heartbeater.cc:503] Master 127.29.174.254:41185 was elected leader, sending a full tablet report...
I20260810 18:50:42.819360 17818 ts_tablet_manager.cc:1428] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260810 18:50:42.823793 17290 tablet.cc:2369] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:42.894814 17825 raft_consensus.cc:482] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:50:42.895330 17825 raft_consensus.cc:504] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.897112 17825 leader_election.cc:288] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf67fb1d74984c2ca6207e8dfa78678a (127.29.174.193:46035), 83e11ae057164ab1b0c7d549cdadba72 (127.29.174.194:34073)
I20260810 18:50:42.904242 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.897731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:38903 (local address 127.29.174.193:46035)
0810 18:50:42.897979 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.897983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.898000 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:50:42.898060 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.898063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.898121 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.898206 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:42.898815 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.899974 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.902084 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.903280 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.903449 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.903526 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.903831 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.903947 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:50:42.904242 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.897350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46035 (local address 127.29.174.195:38903)
0810 18:50:42.897635 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.897656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.897775 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.898246 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.898251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.898282 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:42.898666 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.898679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.900129 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.900133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.901933 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.901946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.903449 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.903456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.903677 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.903703 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.903784 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.903904 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260810 18:50:42.905520 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.897710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:55359 (local address 127.29.174.194:34073)
0810 18:50:42.897907 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.897913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.899150 (+ 1237us) server_negotiation.cc:407] Connection header received
0810 18:50:42.899290 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.899296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.899368 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.899457 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:50:42.900214 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.901054 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.903230 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.904314 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.904460 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.904522 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.905033 (+ 511us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.905166 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:50:42.905609 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.897561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:34073 (local address 127.29.174.195:55359)
0810 18:50:42.899037 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0810 18:50:42.899051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:50:42.899166 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:50:42.899468 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:50:42.899472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:50:42.899488 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:50:42.900045 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.900054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.901218 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.901222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.903049 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:50:42.903061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.904680 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.904686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:50:42.904880 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.904903 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:50:42.904973 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:50:42.905050 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"thread_start_us":1041,"threads_started":1}
I20260810 18:50:42.905550 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68722bf0ccc043528e2cd6af0bd14530" candidate_uuid: "97087064ffcc4fd5b22837545712dc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" is_pre_election: true
I20260810 18:50:42.906414 17222 raft_consensus.cc:2454] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97087064ffcc4fd5b22837545712dc76 in term 0.
I20260810 18:50:42.906663 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68722bf0ccc043528e2cd6af0bd14530" candidate_uuid: "97087064ffcc4fd5b22837545712dc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83e11ae057164ab1b0c7d549cdadba72" is_pre_election: true
I20260810 18:50:42.907253 17586 leader_election.cc:302] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97087064ffcc4fd5b22837545712dc76, bf67fb1d74984c2ca6207e8dfa78678a; no voters:
I20260810 18:50:42.907120 17467 raft_consensus.cc:2454] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97087064ffcc4fd5b22837545712dc76 in term 0.
I20260810 18:50:42.907635 17825 raft_consensus.cc:2781] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:50:42.907773 17825 raft_consensus.cc:482] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:50:42.907938 17825 raft_consensus.cc:3037] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:42.910735 17825 raft_consensus.cc:504] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.911746 17825 leader_election.cc:288] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [CANDIDATE]: Term 1 election: Requested vote from peers bf67fb1d74984c2ca6207e8dfa78678a (127.29.174.193:46035), 83e11ae057164ab1b0c7d549cdadba72 (127.29.174.194:34073)
I20260810 18:50:42.911917 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68722bf0ccc043528e2cd6af0bd14530" candidate_uuid: "97087064ffcc4fd5b22837545712dc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf67fb1d74984c2ca6207e8dfa78678a"
I20260810 18:50:42.912001 17467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68722bf0ccc043528e2cd6af0bd14530" candidate_uuid: "97087064ffcc4fd5b22837545712dc76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83e11ae057164ab1b0c7d549cdadba72"
I20260810 18:50:42.912143 17222 raft_consensus.cc:3037] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:50:42.912256 17467 raft_consensus.cc:3037] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:50:42.913765 17779 tablet.cc:2369] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:42.915058 17467 raft_consensus.cc:2454] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97087064ffcc4fd5b22837545712dc76 in term 1.
I20260810 18:50:42.915134 17222 raft_consensus.cc:2454] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97087064ffcc4fd5b22837545712dc76 in term 1.
I20260810 18:50:42.915583 17585 leader_election.cc:302] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83e11ae057164ab1b0c7d549cdadba72, 97087064ffcc4fd5b22837545712dc76; no voters:
I20260810 18:50:42.915963 17825 raft_consensus.cc:2781] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:50:42.916591 17825 raft_consensus.cc:686] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 1 LEADER]: Becoming Leader. State: Replica: 97087064ffcc4fd5b22837545712dc76, State: Running, Role: LEADER
I20260810 18:50:42.917121 17825 consensus_queue.cc:237] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } }
I20260810 18:50:42.922465 16932 catalog_manager.cc:5495] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97087064ffcc4fd5b22837545712dc76 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "97087064ffcc4fd5b22837545712dc76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97087064ffcc4fd5b22837545712dc76" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 45091 } health_report { overall_health: HEALTHY } } }
W20260810 18:50:42.938223 17547 tablet.cc:2369] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:50:42.945125 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.939357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.29.174.195:45091)
0810 18:50:42.939546 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.939552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:42.939569 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:50:42.939643 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:42.939646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:42.939714 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:42.939832 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:50:42.940253 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.941248 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.942918 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:42.943416 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:42.943528 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:42.944104 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:50:42.944508 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:50:42.944650 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:42.944749 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:50:42.944824 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:50:42.954082 17467 raft_consensus.cc:1261] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 1 FOLLOWER]: Refusing update from remote peer 97087064ffcc4fd5b22837545712dc76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:42.954288 17222 raft_consensus.cc:1261] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 1 FOLLOWER]: Refusing update from remote peer 97087064ffcc4fd5b22837545712dc76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:50:42.955150 17828 consensus_queue.cc:1035] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83e11ae057164ab1b0c7d549cdadba72" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:42.955664 17825 consensus_queue.cc:1035] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf67fb1d74984c2ca6207e8dfa78678a" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:50:42.969264 17834 mvcc.cc:205] Tried to move back new op lower bound from 7317044604730683392 to 7317044604592668672. Current Snapshot: MvccSnapshot[applied={T|T < 7317044604730683392}]
I20260810 18:50:42.989081 16933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:42.989356 16933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:50:42.993288 16933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64e02a0fa80044c7a0006dc212eb2df9: Sending DeleteTablet for 3 replicas of tablet 68722bf0ccc043528e2cd6af0bd14530
I20260810 18:50:42.994715 17434 tablet_service.cc:1514] Processing DeleteTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:42 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:33896
I20260810 18:50:42.994793 17191 tablet_service.cc:1514] Processing DeleteTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:42 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38932
I20260810 18:50:42.995637 17679 tablet_service.cc:1514] Processing DeleteTablet for tablet 68722bf0ccc043528e2cd6af0bd14530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:50:42 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:49722
I20260810 18:50:42.996716 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 17075
I20260810 18:50:43.001624 17856 tablet_replica.cc:330] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: stopping tablet replica
I20260810 18:50:43.002086 17856 raft_consensus.cc:2229] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:50:43.002796 17856 raft_consensus.cc:2258] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:43.009195 17856 ts_tablet_manager.cc:1905] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:43.010252 17852 tablet_replica.cc:330] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: stopping tablet replica
I20260810 18:50:43.010718 17852 raft_consensus.cc:2229] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:50:43.011251 17852 raft_consensus.cc:2258] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:43.012814 17855 tablet_replica.cc:330] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: stopping tablet replica
I20260810 18:50:43.013229 17855 raft_consensus.cc:2229] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:50:43.013262 17852 ts_tablet_manager.cc:1905] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:43.013708 17855 raft_consensus.cc:2258] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:50:43.015727 17855 ts_tablet_manager.cc:1905] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:50:43.020603 17852 ts_tablet_manager.cc:1918] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:43.020821 17852 log.cc:1196] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/68722bf0ccc043528e2cd6af0bd14530
I20260810 18:50:43.021438 17852 ts_tablet_manager.cc:1939] T 68722bf0ccc043528e2cd6af0bd14530 P 83e11ae057164ab1b0c7d549cdadba72: Deleting consensus metadata
I20260810 18:50:43.022470 17855 ts_tablet_manager.cc:1918] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:43.022719 17855 log.cc:1196] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/68722bf0ccc043528e2cd6af0bd14530
I20260810 18:50:43.023432 17855 ts_tablet_manager.cc:1939] T 68722bf0ccc043528e2cd6af0bd14530 P bf67fb1d74984c2ca6207e8dfa78678a: Deleting consensus metadata
I20260810 18:50:43.023375 16902 catalog_manager.cc:4841] TS 83e11ae057164ab1b0c7d549cdadba72 (127.29.174.194:34073): tablet 68722bf0ccc043528e2cd6af0bd14530 (table test-table [id=52faa23b0c14431e8798a334ff2cc08c]) successfully deleted
I20260810 18:50:43.025189 16903 catalog_manager.cc:4841] TS bf67fb1d74984c2ca6207e8dfa78678a (127.29.174.193:46035): tablet 68722bf0ccc043528e2cd6af0bd14530 (table test-table [id=52faa23b0c14431e8798a334ff2cc08c]) successfully deleted
I20260810 18:50:43.025709 17856 ts_tablet_manager.cc:1918] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:50:43.025879 17856 log.cc:1196] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/68722bf0ccc043528e2cd6af0bd14530
I20260810 18:50:43.026516 17856 ts_tablet_manager.cc:1939] T 68722bf0ccc043528e2cd6af0bd14530 P 97087064ffcc4fd5b22837545712dc76: Deleting consensus metadata
I20260810 18:50:43.028208 16905 catalog_manager.cc:4841] TS 97087064ffcc4fd5b22837545712dc76 (127.29.174.195:45091): tablet 68722bf0ccc043528e2cd6af0bd14530 (table test-table [id=52faa23b0c14431e8798a334ff2cc08c]) successfully deleted
Aug 10 18:50:43 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:50:43.039933 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:42.998162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.29.174.193:46035)
0810 18:50:42.998374 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:42.998381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:43.012113 (+ 13732us) server_negotiation.cc:407] Connection header received
0810 18:50:43.012315 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:43.012325 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:43.012416 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:43.012536 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:43.014509 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.015570 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.022595 (+ 7025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.023283 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.023430 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:43.034206 (+ 10776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:43.034218 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:43.034232 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:43.034263 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:43.036913 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.037708 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.037715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.037718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.037845 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.038312 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.038315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.038317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.038723 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:43.039007 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:43.039299 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 18:50:43.039542 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:50:43.040608 17282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:50:43.129354 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 17075
I20260810 18:50:43.155274 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 17297
Aug 10 18:50:43 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:50:43.171496 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:43.156058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.29.174.194:34073)
0810 18:50:43.156261 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:43.156267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:43.156289 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:50:43.156417 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:43.156422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:43.156504 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:43.156588 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:43.157855 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.158679 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.159937 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.160353 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.160467 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:43.164803 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:43.164815 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:43.164830 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:43.164860 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:43.167594 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.168416 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.168423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.168426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.168551 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.168927 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.168930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.168932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.169283 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:43.169500 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:43.169994 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 18:50:43.170868 (+ 874us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:50:43.172271 17533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:50:43.325922 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 17297
I20260810 18:50:43.344949 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 17554
Aug 10 18:50:43 dist-test-slave-6zbq krb5kdc[16114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:50:43.360919 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:50:43.345796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.29.174.195:45091)
0810 18:50:43.345962 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:50:43.345966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:50:43.345986 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:50:43.346178 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:50:43.346182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:50:43.346309 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:50:43.346428 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:50:43.347907 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.348673 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.350037 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:50:43.350545 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:50:43.350678 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:50:43.355154 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:50:43.355166 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:50:43.355181 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:50:43.355232 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:50:43.358132 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.359003 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.359010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.359012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.359171 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:50:43.359579 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:50:43.359582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:50:43.359584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:50:43.359895 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:50:43.360142 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 18:50:43.360476 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 18:50:43.360637 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:50:43.361605 17772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:50:43.479044 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 17554
I20260810 18:50:43.504839 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 16875
I20260810 18:50:43.505952 16989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:50:43.616457 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 16875
I20260810 18:50:43.644798 30395 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:50:43.978904 30395 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:50:43.979197 30395 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:50:44.323217 30395 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:50:44.323 UTC [16170] LOG: received smart shutdown request
2026-08-10 18:50:44.327 UTC [16170] LOG: background worker "logical replication launcher" (PID 16183) exited with exit code 1
2026-08-10 18:50:44.327 UTC [16178] LOG: shutting down
2026-08-10 18:50:44.451 UTC [16170] LOG: database system is shut down
2026-08-10T18:50:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48946 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:50:44 dist-test-slave-6zbq krb5kdc[17881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:50:44 dist-test-slave-6zbq krb5kdc[17881](info): set up 2 sockets
Aug 10 18:50:44 dist-test-slave-6zbq krb5kdc[17881](info): commencing operation
krb5kdc: starting...
W20260810 18:50:45.784442 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:50:45 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:50:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:50:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangeradmin_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangeradmin_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangerlookup_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangerlookup_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
I20260810 18:50:46.098663 30395 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY4kHxe/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/postgres -l logfile start
2026-08-10 18:50:48.229 UTC [17937] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:50:48.230 UTC [17937] LOG: listening on IPv4 address "127.29.174.212", port 47341
2026-08-10 18:50:48.232 UTC [17937] LOG: listening on Unix socket "/tmp/.s.PGSQL.47341"
2026-08-10 18:50:48.256 UTC [17943] LOG: database system was shut down at 2026-08-10 18:50:47 UTC
2026-08-10 18:50:48.259 UTC [17937] LOG: database system is ready to accept connections
I20260810 18:50:49.343546 30395 mini_postgres.cc:96] Postgres bound to 47341
2026-08-10 18:50:49.365 UTC [17961] FATAL: database "slave" does not exist
127.29.174.212:47341 - accepting connections
I20260810 18:50:49.366087 30395 mini_ranger.cc:161] Starting Ranger...
I20260810 18:50:49.392807 30395 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:50:49.759050 30395 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:50:49.759161 30395 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin
2026-08-10 18:50:50,028 [I] DB FLAVOR :POSTGRES
2026-08-10 18:50:50,029 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:50:50,029 [I] Checking connection..
2026-08-10 18:50:50,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:50:50,273 [I] Checking connection passed.
2026-08-10 18:50:50,274 [I] --------- Verifying version history table ---------
2026-08-10 18:50:50,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:50,529 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:50,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:50,788 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:50:50,788 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:50:50,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:50:51.036 UTC [18069] WARNING: there is no transaction in progress
2026-08-10 18:50:51,042 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:50:51,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:50:51,277 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:50:51,277 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:50:51,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:50:51,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:50:51,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-10 18:50:51,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:52,225 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:50:52,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:52,482 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:50:52,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:52,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:52,991 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:50:52,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:50:53.861 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.876 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.890 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.903 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.919 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.937 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.992 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:53.998 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.010 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.019 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.028 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.040 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.049 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.054 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.059 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54.065 UTC [18260] WARNING: there is no transaction in progress
2026-08-10 18:50:54,592 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:50:54,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:50:54,840 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:50:54,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:50:55,096 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:50:55,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:50:55,331 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:50:55,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:50:55,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:50:55,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-10 18:50:56,077 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:50:56,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:50:56,309 [I] DB_PATCHES have already been applied
I20260810 18:50:56.315486 30395 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY4kHxe/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:50:56.315563 30395 mini_ranger.cc:193] Using host: 127.29.174.212
I20260810 18:50:56.334095 30395 mini_ranger.cc:227] Ranger admin URL: http://127.29.174.212:55147
Aug 10, 2026 6:50:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:50:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:50:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:50:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:50:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:50:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:50:57.568256 30395 mini_ranger.cc:160] Time spent starting Ranger: real 8.202s user 0.000s sys 0.050s
Aug 10, 2026 6:51:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:51:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:51:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:51:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:51:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:51:17.545075 30395 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.174.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.174.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangerkms_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.174.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/rangerkms_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.174.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.174.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.174.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/HTTP_127.29.174.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:51:17 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387877, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:51:17.708897 30395 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:51:17.729763 30395 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:51:18.488886 30395 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:51:18.488996 30395 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms
I20260810 18:51:18.489025 30395 mini_ranger_kms.cc:227] Using postgres at 127.29.174.212:47341
2026-08-10 18:51:18,838 [I] DB FLAVOR :POSTGRES
2026-08-10 18:51:18,838 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:51:18,838 [I] Checking connection
2026-08-10 18:51:18,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:51:19,075 [I] connection success
2026-08-10 18:51:19,075 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:51:19,075 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:51:19,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:51:19,317 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:51:19,317 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:51:19,317 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:51:19,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.174.212:47341/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:51:19,577 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:51:20.024969 30395 mini_ranger_kms.cc:310] Created kms service
I20260810 18:51:20.226612 30395 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:51:20.270725 30395 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:51:20 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387877, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:20.630055 30395 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:51:20.630403 30395 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY4kHxe/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY4kHxe/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:51:20.630450 30395 mini_ranger_kms.cc:240] Using host: 127.29.174.212
I20260810 18:51:20.656119 30395 mini_ranger_kms.cc:276] Ranger KMS PID: 18550
I20260810 18:51:20.656244 30395 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.174.212:43389
Aug 10, 2026 6:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:51:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:51:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:51:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:51:22 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387882, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 18:51:23 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387882, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:23.255873 30395 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.547s user 0.075s sys 0.307s
I20260810 18:51:23.256131 30395 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:51:23.256294 30395 mini_ranger_kms.cc:401] 127.29.174.212:43389/kms/v1/keys
Aug 10 18:51:23 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387882, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.174.212@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
Aug 10 18:51:25 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:51:25.779733 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39687
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39687
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:26.092679 18637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:26.092929 18637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:26.093079 18637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:26.100059 18637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:26.100157 18637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:26.100230 18637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:26.100296 18637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:26.100360 18637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:26.109759 18637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.174.212:43389/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39687
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:26.110862 18637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:26.112043 18637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:26.123100 18648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:26.123091 18650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:26.123183 18647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:26.123562 18637 server_base.cc:1034] running on GCE node
I20260810 18:51:26.124257 18637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:26.125468 18637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:26.126660 18637 hybrid_clock.cc:648] HybridClock initialized: now 1786387886126624 us; error 31 us; skew 500 ppm
I20260810 18:51:26.126961 18637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:26 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:26.129771 18637 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:26.131120 18637 webserver.cc:460] Webserver started at http://127.29.174.254:36211/ using document root <none> and password file <none>
I20260810 18:51:26.131649 18637 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:26.131738 18637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:26.131968 18637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:51:26 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387886, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:26.188697 18637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "b1f28a8ab6e74d3c92376b1375081383"
format_stamp: "Formatted at 2026-08-10 18:51:26 on dist-test-slave-6zbq"
server_key: "ec07400d4c89133716568362155fabf9"
server_key_iv: "3510c86ca95c2bb8d51f5531563152fc"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:26.189757 18637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "b1f28a8ab6e74d3c92376b1375081383"
format_stamp: "Formatted at 2026-08-10 18:51:26 on dist-test-slave-6zbq"
server_key: "ec07400d4c89133716568362155fabf9"
server_key_iv: "3510c86ca95c2bb8d51f5531563152fc"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:26.196338 18637 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 18:51:26.215428 18664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:26.217027 18637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:26.217206 18637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "b1f28a8ab6e74d3c92376b1375081383"
format_stamp: "Formatted at 2026-08-10 18:51:26 on dist-test-slave-6zbq"
server_key: "ec07400d4c89133716568362155fabf9"
server_key_iv: "3510c86ca95c2bb8d51f5531563152fc"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:26.217458 18637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:26.272800 18637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:26.276223 18637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:26.276470 18637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:26.293720 18637 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39687
I20260810 18:51:26.293746 18747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39687 every 8 connection(s)
I20260810 18:51:26.294970 18637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:26.298558 18749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:26.299072 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 18637
I20260810 18:51:26.299266 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:51:26.299947 30395 external_mini_cluster.cc:1428] Setting key c62d6a2766a3391d3c7ca9483f7581d3
I20260810 18:51:26.311800 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Bootstrap starting.
Aug 10 18:51:26 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:26.314738 18749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:26.316030 18749 log.cc:824] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:26.318910 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: No bootstrap required, opened a new log
I20260810 18:51:26.322022 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:26.301849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.29.174.254:39687)
0810 18:51:26.302726 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:26.302741 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:26.302789 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:51:26.304612 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:26.304639 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:26.305106 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:26.305529 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:26.306667 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:26.308555 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:26.309799 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:26.310821 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:26.311002 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:26.314586 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:26.314597 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:26.314617 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:26.314670 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:26.318154 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:26.318822 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:26.318828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:26.318834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:26.319022 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:26.319419 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:26.319424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:26.319427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:26.319921 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:26.320191 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:26.320669 (+ 478us) server_negotiation.cc:299] Negotiation successful
0810 18:51:26.321039 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":378,"threads_started":1}
I20260810 18:51:26.326920 18749 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:26.327251 18749 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:26.327345 18749 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f28a8ab6e74d3c92376b1375081383, State: Initialized, Role: FOLLOWER
I20260810 18:51:26.328049 18749 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:26.328307 18760 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:26.328555 18760 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:26.329083 18749 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:26.329138 18760 leader_election.cc:302] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1f28a8ab6e74d3c92376b1375081383; no voters:
I20260810 18:51:26.329270 18749 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:26.329447 18749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:26.329872 18760 leader_election.cc:288] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260810 18:51:26.331624 18749 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:26.331847 18749 leader_election.cc:302] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1f28a8ab6e74d3c92376b1375081383; no voters:
I20260810 18:51:26.331923 18761 raft_consensus.cc:2741] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 18:51:26.332043 18749 leader_election.cc:288] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:51:26.332288 18760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:26.333284 18760 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 LEADER]: Becoming Leader. State: Replica: b1f28a8ab6e74d3c92376b1375081383, State: Running, Role: LEADER
I20260810 18:51:26.333770 18760 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:26.334044 18749 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:26.335830 18761 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1f28a8ab6e74d3c92376b1375081383. Latest consensus state: current_term: 1 leader_uuid: "b1f28a8ab6e74d3c92376b1375081383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } } }
I20260810 18:51:26.336840 18761 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:26.336638 18763 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1f28a8ab6e74d3c92376b1375081383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } } }
I20260810 18:51:26.336974 18763 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:26.337723 18772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:26.343700 18772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:26.346101 18749 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:51:26.360994 18772 catalog_manager.cc:1347] Generated new cluster ID: 08e1356d7c054401911579c8ae70a9f0
I20260810 18:51:26.361150 18772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:26.383199 18772 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:51:26.384618 18772 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:26.392127 18772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Generated new TSK 0
I20260810 18:51:26.392956 18772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:51:27.153256 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:27 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:51:27.594310 18827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:27.594679 18827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:27.594877 18827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:27.605348 18827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:27.605484 18827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:27.605657 18827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:51:27.617812 18827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:27.619647 18827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:27.621446 18827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:27.646593 18847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:27.646760 18851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:27.646900 18848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:27.647827 18827 server_base.cc:1034] running on GCE node
I20260810 18:51:27.648476 18827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:27.649444 18827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:27.650641 18827 hybrid_clock.cc:648] HybridClock initialized: now 1786387887650585 us; error 46 us; skew 500 ppm
I20260810 18:51:27.650957 18827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:27 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:27.654081 18827 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:51:27.655632 18827 webserver.cc:460] Webserver started at http://127.29.174.193:35707/ using document root <none> and password file <none>
I20260810 18:51:27.656299 18827 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:27.656418 18827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:27.656719 18827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:51:27 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:27.677294 18827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "1dd49121e0194addb6494bb400097c30"
format_stamp: "Formatted at 2026-08-10 18:51:27 on dist-test-slave-6zbq"
server_key: "62f3a503c97d677c7d4f0c1213c7bfed"
server_key_iv: "e9dc497204702fd2abf0f0499db81a80"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:27.678318 18827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "1dd49121e0194addb6494bb400097c30"
format_stamp: "Formatted at 2026-08-10 18:51:27 on dist-test-slave-6zbq"
server_key: "62f3a503c97d677c7d4f0c1213c7bfed"
server_key_iv: "e9dc497204702fd2abf0f0499db81a80"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:27.690897 18827 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.006s sys 0.005s
I20260810 18:51:27.728144 18860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:27.730129 18827 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.006s sys 0.008s
I20260810 18:51:27.730454 18827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "1dd49121e0194addb6494bb400097c30"
format_stamp: "Formatted at 2026-08-10 18:51:27 on dist-test-slave-6zbq"
server_key: "62f3a503c97d677c7d4f0c1213c7bfed"
server_key_iv: "e9dc497204702fd2abf0f0499db81a80"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:27.730819 18827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:27.759172 18827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:27.764493 18827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:27.764890 18827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:27.770746 18827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:27.775125 18827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:27.775270 18827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 18:51:27.775446 18827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:27.775552 18827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:51:27 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:27.934353 18827 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:43113
I20260810 18:51:27.934407 19059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:43113 every 8 connection(s)
I20260810 18:51:27.935768 18827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:27.942961 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 18827
I20260810 18:51:27.943198 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:51:27.944113 30395 external_mini_cluster.cc:1428] Setting key 48d98f29e3574d565765263839ed95c7
Aug 10 18:51:27 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:27.992496 19063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:27.966532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49823 (local address 127.29.174.254:39687)
0810 18:51:27.972939 (+ 6407us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:27.972945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:27.972964 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:51:27.973026 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:27.973032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:27.973121 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:27.975324 (+ 2203us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:27.975511 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:27.976693 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:27.979323 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:27.979975 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:27.980105 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:27.984687 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:27.984693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:27.984698 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:27.984750 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:27.988507 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:27.989512 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:27.989518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:27.989523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:27.989689 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:27.990222 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:27.990227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:27.990231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:27.990610 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:27.990751 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:27.991859 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0810 18:51:27.992100 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6274,"thread_start_us":6159,"threads_started":1}
I20260810 18:51:27.993543 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:27.965804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.193:49823)
0810 18:51:27.967709 (+ 1905us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:27.967768 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:27.970173 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:27.973508 (+ 3335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:27.973519 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:27.974163 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:27.975118 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:27.975149 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:27.976889 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:27.976896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:27.979079 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:27.979096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:27.980435 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:27.980443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:27.980729 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:27.981610 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:27.981641 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:27.984399 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:27.988708 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:27.988719 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:27.988739 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:27.989345 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:27.989825 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:27.989830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:27.989833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:27.990085 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:27.990882 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:27.990905 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:27.991350 (+ 445us) client_negotiation.cc:769] Sending connection context
0810 18:51:27.991682 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 18:51:27.992211 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":1314,"threads_started":1}
I20260810 18:51:27.995649 19060 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:27.995956 19060 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:27.996902 19060 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:27.999441 18692 ts_manager.cc:194] Registered new tserver with Master: 1dd49121e0194addb6494bb400097c30 (127.29.174.193:43113)
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
I20260810 18:51:28.001178 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:49823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:51:28.034763 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:28 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387887, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
W20260810 18:51:28.470592 19066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:28.470964 19066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:28.471148 19066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:28.479574 19066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:28.479665 19066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:28.479815 19066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:51:28.489132 19066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:28.490293 19066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:28.491572 19066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:28.505339 19094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:28.505499 19093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:28.505966 19097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:28.507064 19066 server_base.cc:1034] running on GCE node
I20260810 18:51:28.507685 19066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:28.508677 19066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:28.509896 19066 hybrid_clock.cc:648] HybridClock initialized: now 1786387888509841 us; error 52 us; skew 500 ppm
I20260810 18:51:28.510205 19066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:28 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:28.517573 19066 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:51:28.519352 19066 webserver.cc:460] Webserver started at http://127.29.174.194:35761/ using document root <none> and password file <none>
I20260810 18:51:28.520008 19066 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:28.520135 19066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:28.520440 19066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:51:28 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:28.538089 19066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "b88f56138f03443aae7df7408cf222db"
format_stamp: "Formatted at 2026-08-10 18:51:28 on dist-test-slave-6zbq"
server_key: "2196dce22804bad6b43b102dab08098f"
server_key_iv: "b8cb0fede2d82b4d88b82d39286283cf"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:28.538997 19066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "b88f56138f03443aae7df7408cf222db"
format_stamp: "Formatted at 2026-08-10 18:51:28 on dist-test-slave-6zbq"
server_key: "2196dce22804bad6b43b102dab08098f"
server_key_iv: "b8cb0fede2d82b4d88b82d39286283cf"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:28.544739 19066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:51:28.559139 19112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:28.560504 19066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:28.560660 19066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "b88f56138f03443aae7df7408cf222db"
format_stamp: "Formatted at 2026-08-10 18:51:28 on dist-test-slave-6zbq"
server_key: "2196dce22804bad6b43b102dab08098f"
server_key_iv: "b8cb0fede2d82b4d88b82d39286283cf"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:28.560870 19066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:28.587671 19066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:28.591741 19066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:28.592048 19066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:28.593178 19066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:28.594774 19066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:28.594854 19066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:28.594971 19066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:28.595022 19066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:28.636617 19066 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:39161
I20260810 18:51:28.636634 19294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:39161 every 8 connection(s)
I20260810 18:51:28.637909 19066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:28.643679 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 19066
I20260810 18:51:28.643841 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:51:28.644647 30395 external_mini_cluster.cc:1428] Setting key 0bbcf6c8022e90fc9e113a07812223a5
Aug 10 18:51:28 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387888, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:28.669678 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:28.640756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:58189 (local address 127.29.174.254:39687)
0810 18:51:28.650731 (+ 9975us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:28.650737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:28.654273 (+ 3536us) server_negotiation.cc:407] Connection header received
0810 18:51:28.656035 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:28.656041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:28.656118 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:28.656351 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:28.657667 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:28.658644 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:28.660088 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:28.660636 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:28.660754 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:28.664066 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:28.664072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:28.664075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:28.664107 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:28.666814 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:28.667599 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:28.667602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:28.667604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:28.667719 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:28.668163 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:28.668166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:28.668168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:28.668434 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:28.668542 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:28.669283 (+ 741us) server_negotiation.cc:299] Negotiation successful
0810 18:51:28.669445 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9854,"thread_start_us":9716,"threads_started":1}
I20260810 18:51:28.670961 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:28.641391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.194:58189)
0810 18:51:28.654092 (+ 12701us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:28.654137 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:28.655773 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:28.656534 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:28.656542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:28.656955 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:28.657491 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:28.657510 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:28.658773 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:28.658777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:28.659946 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:28.659958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:28.660827 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:28.660832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:28.661031 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:28.661677 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:28.661699 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:28.663843 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:28.666951 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:28.666959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:28.666972 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:28.667497 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:28.667831 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:28.667835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:28.667838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:28.668073 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:28.668557 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:28.668565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:28.668943 (+ 378us) client_negotiation.cc:769] Sending connection context
0810 18:51:28.669215 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 18:51:28.669657 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12437,"thread_start_us":12212,"threads_started":1}
I20260810 18:51:28.672274 19295 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:28.672564 19295 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:28.673378 19295 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:28.674911 18692 ts_manager.cc:194] Registered new tserver with Master: b88f56138f03443aae7df7408cf222db (127.29.174.194:39161)
I20260810 18:51:28.675647 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:58189
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:51:28.706774 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:29.004484 19060 heartbeater.cc:503] Master 127.29.174.254:39687 was elected leader, sending a full tablet report...
W20260810 18:51:29.028087 19301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:29.028359 19301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:29.028501 19301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:29.035813 19301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:29.035928 19301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:29.036072 19301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:51:29.045673 19301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:29.046932 19301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:29.048113 19301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:29.060622 19309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:29.060642 19310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:29.060710 19312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:29.061442 19301 server_base.cc:1034] running on GCE node
I20260810 18:51:29.061966 19301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:29.062870 19301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:29.064093 19301 hybrid_clock.cc:648] HybridClock initialized: now 1786387889064046 us; error 33 us; skew 500 ppm
I20260810 18:51:29.064427 19301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:29.067600 19301 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:51:29.068898 19301 webserver.cc:460] Webserver started at http://127.29.174.195:41855/ using document root <none> and password file <none>
I20260810 18:51:29.069511 19301 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:29.069626 19301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:29.069902 19301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:29.086323 19301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "11228a8b234542fdb373f3a42954a515"
format_stamp: "Formatted at 2026-08-10 18:51:29 on dist-test-slave-6zbq"
server_key: "9dfb143c76ebb17f65f2e55bc0a6b91f"
server_key_iv: "7e18d1b7210e9c9ff52124e7ca5a08d4"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:29.087440 19301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "11228a8b234542fdb373f3a42954a515"
format_stamp: "Formatted at 2026-08-10 18:51:29 on dist-test-slave-6zbq"
server_key: "9dfb143c76ebb17f65f2e55bc0a6b91f"
server_key_iv: "7e18d1b7210e9c9ff52124e7ca5a08d4"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:29.093731 19301 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:51:29.109622 19321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:29.111279 19301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:29.111500 19301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "11228a8b234542fdb373f3a42954a515"
format_stamp: "Formatted at 2026-08-10 18:51:29 on dist-test-slave-6zbq"
server_key: "9dfb143c76ebb17f65f2e55bc0a6b91f"
server_key_iv: "7e18d1b7210e9c9ff52124e7ca5a08d4"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:29.111816 19301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:29.128376 19301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:29.131639 19301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:29.131902 19301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:29.132745 19301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:29.134034 19301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:29.134109 19301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:29.134222 19301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:29.134272 19301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:29.165066 19301 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:46569
I20260810 18:51:29.165129 19517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:46569 every 8 connection(s)
I20260810 18:51:29.166379 19301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:29.167671 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 19301
I20260810 18:51:29.167824 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:51:29.168308 30395 external_mini_cluster.cc:1428] Setting key b7d13e165cc19b554fd8cf71ea8c9335
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:29.184942 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:29.168783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:47879 (local address 127.29.174.254:39687)
0810 18:51:29.168983 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:29.168989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:29.170286 (+ 1297us) server_negotiation.cc:407] Connection header received
0810 18:51:29.172380 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:29.172386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:29.172485 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:29.172569 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:29.173887 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.174705 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.176185 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.176624 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.176734 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:29.179780 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:29.179787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:29.179792 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:29.179830 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:29.182186 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.182937 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.182941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.182943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.183096 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.183580 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.183586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.183590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.183878 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:29.183994 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:29.184542 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 18:51:29.184714 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:29.185750 19520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:29.169357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.195:47879)
0810 18:51:29.170126 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:29.170173 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:29.172127 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:29.172807 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:29.172815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:29.173198 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:29.173725 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:29.173741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.174865 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.174871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:29.176045 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:29.176057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.176839 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.176846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:29.177069 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:29.177621 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:29.177642 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:29.179546 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:29.182315 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:29.182321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:29.182331 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:29.182823 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:29.183213 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:29.183216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:29.183219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:29.183454 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:29.183988 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:29.183994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:29.184261 (+ 267us) client_negotiation.cc:769] Sending connection context
0810 18:51:29.184461 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 18:51:29.184783 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":353,"threads_started":1}
I20260810 18:51:29.186887 19518 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:29.187131 19518 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:29.187731 19518 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:29.188937 18692 ts_manager.cc:194] Registered new tserver with Master: 11228a8b234542fdb373f3a42954a515 (127.29.174.195:46569)
I20260810 18:51:29.189669 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:47879
I20260810 18:51:29.195433 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:29.197053 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 18827
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:51:29.211964 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:29.197900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.29.174.193:43113)
0810 18:51:29.198479 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:29.198485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:29.198508 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:51:29.198558 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:29.198563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:29.198724 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:29.198839 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:29.200183 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.201160 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.202576 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.203132 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.203235 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:29.207159 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:29.207169 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:29.207181 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:29.207213 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:29.209762 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.210281 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.210288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.210291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.210468 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.210774 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.210777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.210779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.211070 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:29.211293 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:29.211509 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 18:51:29.211688 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260810 18:51:29.213109 19052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:29.295738 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 18827
I20260810 18:51:29.315724 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 19066
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:51:29.330837 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:29.316604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.29.174.194:39161)
0810 18:51:29.317104 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:29.317111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:29.317137 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:51:29.317208 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:29.317214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:29.317403 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:29.317528 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:29.318846 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.319846 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.321334 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.321873 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.321966 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:29.326087 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:29.326097 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:29.326110 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:29.326149 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:29.328591 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.329169 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.329176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.329178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.329297 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.329574 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.329576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.329578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.329898 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:29.330146 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:29.330326 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:51:29.330552 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":247,"threads_started":1}
I20260810 18:51:29.332032 19287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:29.412788 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 19066
I20260810 18:51:29.434767 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 19301
Aug 10 18:51:29 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:51:29.451195 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:29.435777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.29.174.195:46569)
0810 18:51:29.436489 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:29.436496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:29.436527 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:51:29.436596 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:29.436603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:29.436812 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:29.436936 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:29.438187 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.439279 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.440496 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:29.441034 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:29.441147 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:29.445323 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:29.445333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:29.445346 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:29.445384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:29.448073 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.449170 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.449177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.449180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.449326 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:29.449757 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:29.449761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:29.449763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:29.450124 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:29.450429 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:29.450697 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 18:51:29.450902 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":389,"threads_started":1}
I20260810 18:51:29.452418 19513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:29.532900 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 19301
I20260810 18:51:29.551558 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 18637
I20260810 18:51:29.552791 18743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:29.655107 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 18637
I20260810 18:51:29.684227 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39687
--webserver_interface=127.29.174.254
--webserver_port=36211
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39687
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:29.999967 19538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:30.000213 19538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:30.000298 19538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:30.007172 19538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:30.007263 19538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:30.007311 19538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:30.007365 19538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:30.007409 19538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:30.016703 19538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39687
--ranger_config_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=36211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:30.017705 19538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:30.018867 19538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:30.029870 19544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:30.029902 19545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:30.029829 19547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:30.030692 19538 server_base.cc:1034] running on GCE node
I20260810 18:51:30.031376 19538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:30.032675 19538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:30.033893 19538 hybrid_clock.cc:648] HybridClock initialized: now 1786387890033834 us; error 41 us; skew 500 ppm
I20260810 18:51:30.034217 19538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:30 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:30.037097 19538 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:30.038489 19538 webserver.cc:460] Webserver started at http://127.29.174.254:36211/ using document root <none> and password file <none>
I20260810 18:51:30.039067 19538 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:30.039191 19538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:51:30 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:30.823192 19538 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:51:30.826004 19556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:30.827375 19538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:30.827543 19538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "b1f28a8ab6e74d3c92376b1375081383"
format_stamp: "Formatted at 2026-08-10 18:51:26 on dist-test-slave-6zbq"
server_key: "ec07400d4c89133716568362155fabf9"
server_key_iv: "3510c86ca95c2bb8d51f5531563152fc"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:30.828503 19538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:30.864874 19538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:30.868213 19538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:30.868468 19538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:30.885813 19538 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39687
I20260810 18:51:30.885836 19650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39687 every 8 connection(s)
I20260810 18:51:30.887208 19538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:30.888098 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 19538
I20260810 18:51:30.888984 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:43113
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:39687
--webserver_port=35707
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:30.893914 19652 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:30.897684 19652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Bootstrap starting.
I20260810 18:51:30.920593 19652 log.cc:824] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:30.929085 19652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:30.929560 19652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Bootstrap complete.
I20260810 18:51:30.938411 19652 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:30.939211 19652 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1f28a8ab6e74d3c92376b1375081383, State: Initialized, Role: FOLLOWER
I20260810 18:51:30.939754 19652 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:30.939967 19652 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:30.940066 19652 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:30.940222 19652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:51:30.943768 19652 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:30.944201 19652 leader_election.cc:302] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1f28a8ab6e74d3c92376b1375081383; no voters:
I20260810 18:51:30.944754 19652 leader_election.cc:288] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:51:30.944895 19658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:51:30.945907 19658 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [term 2 LEADER]: Becoming Leader. State: Replica: b1f28a8ab6e74d3c92376b1375081383, State: Running, Role: LEADER
I20260810 18:51:30.946502 19658 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } }
I20260810 18:51:30.946800 19652 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:30.949234 19660 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1f28a8ab6e74d3c92376b1375081383. Latest consensus state: current_term: 2 leader_uuid: "b1f28a8ab6e74d3c92376b1375081383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } } }
I20260810 18:51:30.949244 19659 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b1f28a8ab6e74d3c92376b1375081383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1f28a8ab6e74d3c92376b1375081383" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39687 } } }
I20260810 18:51:30.949671 19660 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:30.949676 19659 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383 [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:30.955603 19666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:30.962324 19666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:30.963174 19652 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:51:30.963901 19666 catalog_manager.cc:1259] Loaded cluster ID: 08e1356d7c054401911579c8ae70a9f0
I20260810 18:51:30.964063 19666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:30.967968 19666 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:30.969401 19666 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Loaded TSK: 0
I20260810 18:51:30.970973 19666 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:51:31.321664 19655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:31.322091 19655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:31.322300 19655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:31.333307 19655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:31.333439 19655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:31.333642 19655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:51:31.348757 19655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:43113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=35707
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:31.350353 19655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:31.352109 19655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:31.394367 19722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:31.394268 19721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:31.399153 19725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:31.400032 19655 server_base.cc:1034] running on GCE node
I20260810 18:51:31.400892 19655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:31.414598 19655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:31.415957 19655 hybrid_clock.cc:648] HybridClock initialized: now 1786387891415879 us; error 62 us; skew 500 ppm
I20260810 18:51:31.416422 19655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:31 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387891, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:31.421065 19655 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:51:31.443732 19655 webserver.cc:460] Webserver started at http://127.29.174.193:35707/ using document root <none> and password file <none>
I20260810 18:51:31.444567 19655 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:31.444722 19655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:51:31 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387891, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:31.499881 19655 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.001s
I20260810 18:51:31.503968 19744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:31.505739 19655 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260810 18:51:31.505911 19655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "1dd49121e0194addb6494bb400097c30"
format_stamp: "Formatted at 2026-08-10 18:51:27 on dist-test-slave-6zbq"
server_key: "62f3a503c97d677c7d4f0c1213c7bfed"
server_key_iv: "e9dc497204702fd2abf0f0499db81a80"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:31.507285 19655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:31.541910 19655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:31.547155 19655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:31.547461 19655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:31.548692 19655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:31.550681 19655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:31.550797 19655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:31.550957 19655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:31.551049 19655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:31.597497 19655 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:43113
I20260810 18:51:31.597555 19945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:43113 every 8 connection(s)
I20260810 18:51:31.599006 19655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:31.602219 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 19655
I20260810 18:51:31.603070 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:39161
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:39687
--webserver_port=35761
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:31 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387891, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:31.633394 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:31.605346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.193:42987)
0810 18:51:31.606308 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:31.606380 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:31.608749 (+ 2369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:31.611154 (+ 2405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:31.611165 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:31.611805 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:31.612569 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:31.612610 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:31.615023 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:31.615029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:31.616953 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:31.616970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:31.618729 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:31.618738 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:31.619006 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:31.619760 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:31.619796 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:31.622483 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:31.626911 (+ 4428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:31.626920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:31.626936 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:31.627551 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:31.628114 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:31.628119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:31.628122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:31.628397 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:31.629514 (+ 1117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:31.629522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:31.629940 (+ 418us) client_negotiation.cc:769] Sending connection context
0810 18:51:31.630226 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 18:51:31.630792 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":545,"threads_started":1}
I20260810 18:51:31.635141 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:31.605287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:42987 (local address 127.29.174.254:39687)
0810 18:51:31.605993 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:31.606012 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:31.606772 (+ 760us) server_negotiation.cc:407] Connection header received
0810 18:51:31.609547 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:31.609568 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:31.610043 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:31.610435 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:31.612832 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:31.614797 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:31.617132 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:31.618147 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:31.618321 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:31.622788 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:31.622799 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:31.622818 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:31.622864 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:31.626734 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:31.627746 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:31.627755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:31.627762 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:31.627945 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:31.628546 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:31.628552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:31.628555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:31.629229 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:31.629433 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:31.632244 (+ 2811us) server_negotiation.cc:299] Negotiation successful
0810 18:51:31.632665 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":258,"threads_started":1}
I20260810 18:51:31.638778 19946 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:31.639101 19946 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:31.639990 19946 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
W20260810 18:51:31.643383 19946 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:39687 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:51:31.643603 19946 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:31.643864 19946 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:31.645119 19946 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:31.645453 19946 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
W20260810 18:51:31.646551 19946 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:51:32.044885 19954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:32.045228 19954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:32.045429 19954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:32.058705 19954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:32.058905 19954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:32.059127 19954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:51:32.075500 19954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:39161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=35761
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:32.077208 19954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:32.078938 19954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:32.103549 19984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:32.107559 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:32.108332 19983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:32.109145 19954 server_base.cc:1034] running on GCE node
I20260810 18:51:32.109808 19954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:32.110982 19954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:32.112269 19954 hybrid_clock.cc:648] HybridClock initialized: now 1786387892112184 us; error 69 us; skew 500 ppm
I20260810 18:51:32.112682 19954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:32.116890 19954 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:51:32.134797 19954 webserver.cc:460] Webserver started at http://127.29.174.194:35761/ using document root <none> and password file <none>
I20260810 18:51:32.135644 19954 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:32.135855 19954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:32.173405 19954 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.001s
I20260810 18:51:32.177070 20002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:32.178979 19954 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:51:32.179205 19954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "b88f56138f03443aae7df7408cf222db"
format_stamp: "Formatted at 2026-08-10 18:51:28 on dist-test-slave-6zbq"
server_key: "2196dce22804bad6b43b102dab08098f"
server_key_iv: "b8cb0fede2d82b4d88b82d39286283cf"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:32.180713 19954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:32.219465 19954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:32.223764 19954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:32.224020 19954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:32.225318 19954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:32.227108 19954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:32.227236 19954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:32.227351 19954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:32.227428 19954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:32.281409 19954 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:39161
I20260810 18:51:32.281468 20187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:39161 every 8 connection(s)
I20260810 18:51:32.283669 19954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:32.290968 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 19954
I20260810 18:51:32.291764 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.174.212:43389/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:46569
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:39687
--webserver_port=41855
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:32.366151 20194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:32.294828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:55575 (local address 127.29.174.254:39687)
0810 18:51:32.339199 (+ 44371us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:32.339206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:32.339233 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:51:32.342127 (+ 2894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:32.342135 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:32.342229 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:32.342328 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:32.349494 (+ 7166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:32.350829 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:32.353235 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:32.353887 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:32.353983 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:32.358496 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:32.358504 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:32.358509 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:32.358552 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:32.362236 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:32.363254 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:32.363260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:32.363264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:32.363424 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:32.364006 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:32.364011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:32.364015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:32.364378 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:32.364510 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:32.365508 (+ 998us) server_negotiation.cc:299] Negotiation successful
0810 18:51:32.365761 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44227,"thread_start_us":47098,"threads_started":1}
I20260810 18:51:32.368523 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:32.295242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.194:55575)
0810 18:51:32.333841 (+ 38599us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:32.333926 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:32.341625 (+ 7699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:32.346880 (+ 5255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:32.346895 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:32.347672 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:32.349232 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:32.349263 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:32.351049 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:32.351055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:32.353060 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:32.353078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:32.354376 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:32.354387 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:32.354709 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:32.355470 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:32.355509 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:32.358203 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:32.362443 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:32.362453 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:32.362471 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:32.363097 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:32.363578 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:32.363583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:32.363587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:32.363870 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:32.364607 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:32.364616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:32.365060 (+ 444us) client_negotiation.cc:769] Sending connection context
0810 18:51:32.366529 (+ 1469us) client_negotiation.cc:240] Negotiation successful
0810 18:51:32.367117 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38164,"thread_start_us":42516,"threads_started":1}
I20260810 18:51:32.370422 20189 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:32.370780 20189 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:32.371711 20189 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:32.373984 19594 ts_manager.cc:194] Registered new tserver with Master: b88f56138f03443aae7df7408cf222db (127.29.174.194:39161)
I20260810 18:51:32.376255 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:55575
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:32.646732 19946 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:32.647176 19946 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:32.648418 19594 ts_manager.cc:194] Registered new tserver with Master: 1dd49121e0194addb6494bb400097c30 (127.29.174.193:43113)
I20260810 18:51:32.649518 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:42987
W20260810 18:51:32.794407 20196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:32.794806 20196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:32.794997 20196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:32.805943 20196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:32.806130 20196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:32.806313 20196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:32.825310 20196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.174.212:43389/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:46569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=41855
--tserver_master_addrs=127.29.174.254:39687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:32.827158 20196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:32.828837 20196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:32.882259 20211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:32.890725 20208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:32.891603 20214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:32.892119 20196 server_base.cc:1034] running on GCE node
I20260810 18:51:32.892894 20196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:32.906596 20196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:32.907833 20196 hybrid_clock.cc:648] HybridClock initialized: now 1786387892907749 us; error 78 us; skew 500 ppm
I20260810 18:51:32.908165 20196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:32.911749 20196 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:51:32.913841 20196 webserver.cc:460] Webserver started at http://127.29.174.195:41855/ using document root <none> and password file <none>
I20260810 18:51:32.914736 20196 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:32.914923 20196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:51:32 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for HTTP/127.29.174.212@KRBTEST.COM
I20260810 18:51:32.948066 20196 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.008s
I20260810 18:51:32.960351 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:32.962077 20196 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.008s sys 0.001s
I20260810 18:51:32.962258 20196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "11228a8b234542fdb373f3a42954a515"
format_stamp: "Formatted at 2026-08-10 18:51:29 on dist-test-slave-6zbq"
server_key: "9dfb143c76ebb17f65f2e55bc0a6b91f"
server_key_iv: "7e18d1b7210e9c9ff52124e7ca5a08d4"
server_key_version: "kuduclusterkey@0"
I20260810 18:51:32.963521 20196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:32.993089 20196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:32.997188 20196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:32.997444 20196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:33.002803 20196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:33.004237 20196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:33.004336 20196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:33.004447 20196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:33.004518 20196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:33.059680 20196 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:46569
I20260810 18:51:33.059850 20440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:46569 every 8 connection(s)
I20260810 18:51:33.061177 20196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:33.066797 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 20196
Aug 10 18:51:33 dist-test-slave-6zbq krb5kdc[17881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:33.091174 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.067887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.29.174.254:39687)
0810 18:51:33.071066 (+ 3179us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.071069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.074840 (+ 3771us) server_negotiation.cc:407] Connection header received
0810 18:51:33.075048 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.075053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.075139 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.075233 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.077272 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.078454 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.080528 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.081024 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.081158 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.084324 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.084330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.084335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.084377 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.088007 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.088762 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.088768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.088771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.088919 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.089535 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.089540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.089544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.089895 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.090206 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.090547 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.090819 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":3112,"thread_start_us":2601,"threads_started":1}
I20260810 18:51:33.095980 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.064620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39687 (local address 127.29.174.195:49795)
0810 18:51:33.075845 (+ 11225us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.075900 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.078012 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.078776 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.078790 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.079382 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:33.080127 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.080153 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.081870 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.081875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.083168 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.083181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.083971 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.083979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.084183 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.084924 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:33.084954 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:33.087328 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:33.091596 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:33.091604 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:33.091620 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:33.092566 (+ 946us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:33.093052 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:33.093057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:33.093060 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:33.093331 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:33.093946 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:33.093955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:33.094273 (+ 318us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.094591 (+ 318us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.094998 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10927,"thread_start_us":10731,"threads_started":1}
I20260810 18:51:33.096930 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.064129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:49795 (local address 127.29.174.254:39687)
0810 18:51:33.070749 (+ 6620us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.070757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.076009 (+ 5252us) server_negotiation.cc:407] Connection header received
0810 18:51:33.078259 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.078264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.078381 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.078482 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.080357 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.081691 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.083323 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.083741 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.083849 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.087605 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.087612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.087617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.087672 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.091433 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.092697 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.092703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.092716 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.092924 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.093472 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.093477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.093481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.093724 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.093858 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.096497 (+ 2639us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.096676 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":6479,"thread_start_us":6739,"threads_started":1}
I20260810 18:51:33.098073 20444 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39687
I20260810 18:51:33.098431 20444 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:33.099239 20444 heartbeater.cc:511] Master 127.29.174.254:39687 requested a full tablet report, sending...
I20260810 18:51:33.100720 19594 ts_manager.cc:194] Registered new tserver with Master: 11228a8b234542fdb373f3a42954a515 (127.29.174.195:46569)
I20260810 18:51:33.101501 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:49795
I20260810 18:51:33.106940 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:33.124993 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.110170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.29.174.254:39687)
0810 18:51:33.110403 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.110411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.110799 (+ 388us) server_negotiation.cc:407] Connection header received
0810 18:51:33.110979 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.110983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.111052 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.111125 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.113112 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.113805 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.115787 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.116212 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.116307 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.119495 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.119502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.119506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.119544 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.122218 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.123023 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.123027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.123029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.123147 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.123529 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.123532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.123534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.123791 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.123961 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.124415 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.124654 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:51:33.129839 19594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:33.137492 19594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:33.167182 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.151794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:46569 (local address 127.0.0.1:35128)
0810 18:51:33.154040 (+ 2246us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.154074 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.154255 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.155290 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.155299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.156418 (+ 1119us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:33.156889 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.156904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.160234 (+ 3330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.160241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.162796 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.162814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.166176 (+ 3362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.166183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.166437 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.166491 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.166618 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.166811 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2048,"threads_started":1}
I20260810 18:51:33.169469 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.152443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.29.174.195:46569)
0810 18:51:33.153608 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.153614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.154183 (+ 569us) server_negotiation.cc:407] Connection header received
0810 18:51:33.154477 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.154485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.154741 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.154915 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:33.157115 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.159965 (+ 2850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.164054 (+ 4089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.165625 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.165821 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.165997 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.167646 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.167827 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":905,"threads_started":1}
I20260810 18:51:33.178861 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.152338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.29.174.193:43113)
0810 18:51:33.152964 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.152973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.167533 (+ 14560us) server_negotiation.cc:407] Connection header received
0810 18:51:33.169775 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.169784 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.170038 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.170202 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:33.170931 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.172598 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.175966 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.177451 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.177612 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.177766 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.178303 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.178492 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":687,"threads_started":1}
I20260810 18:51:33.179569 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.151801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38736)
0810 18:51:33.167422 (+ 15621us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.167441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.169314 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.170557 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.170562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.170589 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:33.170754 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.170764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.172787 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.172792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.175779 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.175795 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.177908 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.177914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.178098 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.178132 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.179047 (+ 915us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.179196 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15512,"mutex_wait_us":869,"thread_start_us":1071,"threads_started":1}
I20260810 18:51:33.184181 20324 tablet_service.cc:1467] Processing CreateTablet for tablet 1318a1008fe84acab5673432e236a54d (DEFAULT_TABLE table=test-table [id=c3cf9b6e121f48f8a8181683a75aef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:33.184181 19830 tablet_service.cc:1467] Processing CreateTablet for tablet 1318a1008fe84acab5673432e236a54d (DEFAULT_TABLE table=test-table [id=c3cf9b6e121f48f8a8181683a75aef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:33.185508 20324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318a1008fe84acab5673432e236a54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:33.185626 19830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318a1008fe84acab5673432e236a54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:33.189142 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.154577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:39161 (local address 127.0.0.1:53726)
0810 18:51:33.179544 (+ 24967us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.179565 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.180475 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.182456 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.182461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.182486 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:33.182802 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.182814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.184662 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.184667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.186628 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.186643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.188428 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.188434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.188608 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.188642 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.188726 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.188845 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24832,"thread_start_us":25459,"threads_started":1}
I20260810 18:51:33.189301 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.154832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.29.174.194:39161)
0810 18:51:33.180074 (+ 25242us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.180083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.180110 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:51:33.180676 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.180684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.181960 (+ 1276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.182155 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:33.183005 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.184490 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.186821 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.188191 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.188367 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.188526 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.188820 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.188981 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25107,"thread_start_us":25004,"threads_started":1}
I20260810 18:51:33.197575 20475 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Bootstrap starting.
I20260810 18:51:33.197376 20082 tablet_service.cc:1467] Processing CreateTablet for tablet 1318a1008fe84acab5673432e236a54d (DEFAULT_TABLE table=test-table [id=c3cf9b6e121f48f8a8181683a75aef3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:33.198894 20082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1318a1008fe84acab5673432e236a54d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:33.199445 20479 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Bootstrap starting.
I20260810 18:51:33.201778 20475 tablet_bootstrap.cc:654] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:33.202826 20479 tablet_bootstrap.cc:654] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:33.203051 20475 log.cc:824] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:33.203951 20479 log.cc:824] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:33.206113 20475 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: No bootstrap required, opened a new log
I20260810 18:51:33.206413 20475 ts_tablet_manager.cc:1397] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260810 18:51:33.208889 20479 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: No bootstrap required, opened a new log
I20260810 18:51:33.209123 20479 ts_tablet_manager.cc:1397] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 18:51:33.211143 20488 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Bootstrap starting.
I20260810 18:51:33.214740 20488 tablet_bootstrap.cc:654] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:33.215651 20488 log.cc:824] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:33.217072 20475 raft_consensus.cc:348] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.217491 20475 raft_consensus.cc:374] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:33.217586 20475 raft_consensus.cc:729] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd49121e0194addb6494bb400097c30, State: Initialized, Role: FOLLOWER
I20260810 18:51:33.218125 20488 tablet_bootstrap.cc:492] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: No bootstrap required, opened a new log
I20260810 18:51:33.218389 20488 ts_tablet_manager.cc:1397] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:51:33.218183 20475 consensus_queue.cc:260] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.219309 19946 heartbeater.cc:503] Master 127.29.174.254:39687 was elected leader, sending a full tablet report...
I20260810 18:51:33.219859 20475 ts_tablet_manager.cc:1428] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 18:51:33.220698 20479 raft_consensus.cc:348] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.221101 20479 raft_consensus.cc:374] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:33.221199 20479 raft_consensus.cc:729] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11228a8b234542fdb373f3a42954a515, State: Initialized, Role: FOLLOWER
I20260810 18:51:33.221689 20479 consensus_queue.cc:260] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.226862 20488 raft_consensus.cc:348] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.227252 20488 raft_consensus.cc:374] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:33.227377 20488 raft_consensus.cc:729] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88f56138f03443aae7df7408cf222db, State: Initialized, Role: FOLLOWER
I20260810 18:51:33.227845 20488 consensus_queue.cc:260] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.228870 20189 heartbeater.cc:503] Master 127.29.174.254:39687 was elected leader, sending a full tablet report...
I20260810 18:51:33.229602 20479 ts_tablet_manager.cc:1428] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
I20260810 18:51:33.229713 20488 ts_tablet_manager.cc:1428] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260810 18:51:33.230046 20444 heartbeater.cc:503] Master 127.29.174.254:39687 was elected leader, sending a full tablet report...
I20260810 18:51:33.308492 20493 raft_consensus.cc:482] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:33.308755 20493 raft_consensus.cc:504] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.310262 20493 leader_election.cc:288] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dd49121e0194addb6494bb400097c30 (127.29.174.193:43113), b88f56138f03443aae7df7408cf222db (127.29.174.194:39161)
I20260810 18:51:33.316599 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.310774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:57497 (local address 127.29.174.193:43113)
0810 18:51:33.311042 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.311047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.311066 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:51:33.311137 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.311142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.311212 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.311298 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:33.311798 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.312858 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.314720 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.315797 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.315921 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.315975 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.316289 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.316380 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:51:33.316669 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.310577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.29.174.195:57497)
0810 18:51:33.310730 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.310748 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.310859 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.311470 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.311474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.311501 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:33.311648 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.311655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.313010 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.313016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.314562 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.314574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.315948 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.315954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.316136 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.316172 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.316269 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.316397 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 18:51:33.317335 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.311159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:40905 (local address 127.29.174.194:39161)
0810 18:51:33.311369 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.311375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.311724 (+ 349us) server_negotiation.cc:407] Connection header received
0810 18:51:33.311911 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.311914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.311972 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.312055 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:33.312561 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.313461 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.315445 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.316400 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.316536 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.316602 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.316964 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.317089 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:51:33.317363 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.311005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:39161 (local address 127.29.174.195:40905)
0810 18:51:33.311625 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.311639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:33.311771 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:33.312078 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:33.312083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:33.312117 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:33.312430 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.312441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.313590 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.313594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.315272 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:33.315289 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.316600 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.316614 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:33.316798 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.316830 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:51:33.316918 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:51:33.317036 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":298,"threads_started":1}
I20260810 18:51:33.317647 19865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318a1008fe84acab5673432e236a54d" candidate_uuid: "11228a8b234542fdb373f3a42954a515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd49121e0194addb6494bb400097c30" is_pre_election: true
I20260810 18:51:33.318117 19865 raft_consensus.cc:2454] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11228a8b234542fdb373f3a42954a515 in term 0.
I20260810 18:51:33.318262 20116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318a1008fe84acab5673432e236a54d" candidate_uuid: "11228a8b234542fdb373f3a42954a515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88f56138f03443aae7df7408cf222db" is_pre_election: true
W20260810 18:51:33.318775 20446 tablet.cc:2369] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:33.318797 20116 raft_consensus.cc:2454] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11228a8b234542fdb373f3a42954a515 in term 0.
I20260810 18:51:33.318806 20225 leader_election.cc:302] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11228a8b234542fdb373f3a42954a515, 1dd49121e0194addb6494bb400097c30; no voters:
I20260810 18:51:33.319197 20493 raft_consensus.cc:2781] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:33.319406 20493 raft_consensus.cc:482] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:33.319532 20493 raft_consensus.cc:3037] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:33.321601 20493 raft_consensus.cc:504] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.322104 20493 leader_election.cc:288] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [CANDIDATE]: Term 1 election: Requested vote from peers 1dd49121e0194addb6494bb400097c30 (127.29.174.193:43113), b88f56138f03443aae7df7408cf222db (127.29.174.194:39161)
I20260810 18:51:33.322523 19865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318a1008fe84acab5673432e236a54d" candidate_uuid: "11228a8b234542fdb373f3a42954a515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dd49121e0194addb6494bb400097c30"
I20260810 18:51:33.322585 20116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1318a1008fe84acab5673432e236a54d" candidate_uuid: "11228a8b234542fdb373f3a42954a515" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88f56138f03443aae7df7408cf222db"
I20260810 18:51:33.322688 19865 raft_consensus.cc:3037] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:33.322765 20116 raft_consensus.cc:3037] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:33.324777 19865 raft_consensus.cc:2454] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11228a8b234542fdb373f3a42954a515 in term 1.
I20260810 18:51:33.324892 20116 raft_consensus.cc:2454] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11228a8b234542fdb373f3a42954a515 in term 1.
I20260810 18:51:33.325121 20225 leader_election.cc:302] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11228a8b234542fdb373f3a42954a515, 1dd49121e0194addb6494bb400097c30; no voters:
I20260810 18:51:33.325515 20493 raft_consensus.cc:2781] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:33.325938 20493 raft_consensus.cc:686] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 1 LEADER]: Becoming Leader. State: Replica: 11228a8b234542fdb373f3a42954a515, State: Running, Role: LEADER
I20260810 18:51:33.326360 20493 consensus_queue.cc:237] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } }
I20260810 18:51:33.330493 19594 catalog_manager.cc:5495] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11228a8b234542fdb373f3a42954a515 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "11228a8b234542fdb373f3a42954a515" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11228a8b234542fdb373f3a42954a515" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 } health_report { overall_health: UNKNOWN } } }
W20260810 18:51:33.349766 20193 tablet.cc:2369] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:33.351416 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.346030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.29.174.195:46569)
0810 18:51:33.346219 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.346225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.346244 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:51:33.346338 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.346378 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.346478 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.346614 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:33.346999 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.347859 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.349259 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.349754 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.349888 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.350440 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:33.350933 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:33.351037 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.351112 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.351176 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:51:33.354975 19951 tablet.cc:2369] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:33.358997 19865 raft_consensus.cc:1261] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 1 FOLLOWER]: Refusing update from remote peer 11228a8b234542fdb373f3a42954a515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:33.359058 20116 raft_consensus.cc:1261] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 1 FOLLOWER]: Refusing update from remote peer 11228a8b234542fdb373f3a42954a515: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:33.359833 20493 consensus_queue.cc:1035] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dd49121e0194addb6494bb400097c30" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:33.360100 20496 consensus_queue.cc:1035] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b88f56138f03443aae7df7408cf222db" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 39161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:33.385133 19588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:33.385452 19588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:33.389098 19588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1f28a8ab6e74d3c92376b1375081383: Sending DeleteTablet for 3 replicas of tablet 1318a1008fe84acab5673432e236a54d
I20260810 18:51:33.390146 20324 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318a1008fe84acab5673432e236a54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:33 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:35128
I20260810 18:51:33.390195 19830 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318a1008fe84acab5673432e236a54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:33 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38736
I20260810 18:51:33.391108 20082 tablet_service.cc:1514] Processing DeleteTablet for tablet 1318a1008fe84acab5673432e236a54d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:33 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:53726
I20260810 18:51:33.399948 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 19655
I20260810 18:51:33.401109 20517 tablet_replica.cc:330] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: stopping tablet replica
I20260810 18:51:33.401646 20517 raft_consensus.cc:2229] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:51:33.402503 20517 raft_consensus.cc:2258] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:33.405128 20517 ts_tablet_manager.cc:1905] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:33.412477 20517 ts_tablet_manager.cc:1918] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:33.412729 20517 log.cc:1196] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/1318a1008fe84acab5673432e236a54d
I20260810 18:51:33.413789 20517 ts_tablet_manager.cc:1939] T 1318a1008fe84acab5673432e236a54d P 11228a8b234542fdb373f3a42954a515: Deleting consensus metadata
I20260810 18:51:33.416116 19561 catalog_manager.cc:4841] TS 11228a8b234542fdb373f3a42954a515 (127.29.174.195:46569): tablet 1318a1008fe84acab5673432e236a54d (table test-table [id=c3cf9b6e121f48f8a8181683a75aef3a]) successfully deleted
I20260810 18:51:33.417475 20519 tablet_replica.cc:330] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: stopping tablet replica
I20260810 18:51:33.417966 20519 raft_consensus.cc:2229] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:33.418787 20518 tablet_replica.cc:330] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30: stopping tablet replica
I20260810 18:51:33.419294 20519 raft_consensus.cc:2258] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:33.419749 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.401331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.29.174.193:43113)
0810 18:51:33.401503 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.401509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.401527 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:51:33.401582 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.401587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.401660 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.401756 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.403636 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.404768 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.406685 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.407332 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.407442 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.410936 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.410951 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.410968 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.411011 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.415682 (+ 4671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.416646 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.416657 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.416661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.416835 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.417355 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.417362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.417366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.417971 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.418423 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.419098 (+ 675us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.419337 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:51:33.419864 20518 raft_consensus.cc:2229] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:33.420532 20518 raft_consensus.cc:2258] T 1318a1008fe84acab5673432e236a54d P 1dd49121e0194addb6494bb400097c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:33.420892 19939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:33.423921 20519 ts_tablet_manager.cc:1905] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:33.431902 20519 ts_tablet_manager.cc:1918] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:33.432118 20519 log.cc:1196] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/1318a1008fe84acab5673432e236a54d
I20260810 18:51:33.432775 20519 ts_tablet_manager.cc:1939] T 1318a1008fe84acab5673432e236a54d P b88f56138f03443aae7df7408cf222db: Deleting consensus metadata
I20260810 18:51:33.434765 19559 catalog_manager.cc:4841] TS b88f56138f03443aae7df7408cf222db (127.29.174.194:39161): tablet 1318a1008fe84acab5673432e236a54d (table test-table [id=c3cf9b6e121f48f8a8181683a75aef3a]) successfully deleted
I20260810 18:51:33.519282 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 19655
I20260810 18:51:33.543792 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 19954
W20260810 18:51:33.543910 19559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:43113 (error 108)
I20260810 18:51:33.546535 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.545983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38756)
0810 18:51:33.546218 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.546372 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260810 18:51:33.548832 19559 catalog_manager.cc:4584] TS 1dd49121e0194addb6494bb400097c30 (127.29.174.193:43113): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1318a1008fe84acab5673432e236a54d: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
I20260810 18:51:33.564282 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.545517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53738 (local address 127.29.174.194:39161)
0810 18:51:33.545701 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.545707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.545740 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:51:33.545853 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.545868 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.545987 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.546120 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.548438 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.549754 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.551141 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.551714 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.551853 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.558895 (+ 7042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.558913 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.558929 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.558962 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.561649 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.562272 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.562286 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.562288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.562469 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.562828 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.562833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.562836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.563183 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.563476 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.563744 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.563974 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:51:33.564940 20181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:33.602452 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.601858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38764)
0810 18:51:33.602100 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.602240 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:51:33.603749 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.603240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38772)
0810 18:51:33.603498 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.603585 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260810 18:51:33.652426 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 19954
I20260810 18:51:33.664503 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.663920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38778)
0810 18:51:33.664170 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.664287 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 18:51:33.665771 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.665377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38782)
0810 18:51:33.665543 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.665636 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:51:33.674163 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 20196
I20260810 18:51:33.688462 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.674850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.29.174.195:46569)
0810 18:51:33.675038 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:33.675044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:33.675063 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:51:33.675184 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:33.675190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:33.675259 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:33.675355 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:33.676634 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.677425 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.678717 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:33.679160 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:33.679296 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:33.681741 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:33.681755 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:33.681775 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:33.681830 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:33.685756 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.686401 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.686410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.686414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.686596 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:33.686937 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:33.686943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:33.686946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:33.687396 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:33.687706 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:33.687863 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 18:51:33.688075 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:51:33.689102 20434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:33.746536 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.745935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38788)
0810 18:51:33.746165 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.746297 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 18:51:33.747635 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.747184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38790)
0810 18:51:33.747408 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.747499 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 18:51:33.808771 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 20196
I20260810 18:51:33.833011 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 19538
I20260810 18:51:33.834227 19646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:33.947540 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:33.947046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:43113 (local address 127.0.0.1:38794)
0810 18:51:33.947224 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:33.947312 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:51:33.948096 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 19538
I20260810 18:51:33.974843 30395 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:51:34.308848 30395 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:51:34.309077 30395 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:51:34.653542 30395 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:51:34.653 UTC [17937] LOG: received smart shutdown request
2026-08-10 18:51:34.656 UTC [17937] LOG: background worker "logical replication launcher" (PID 17955) exited with exit code 1
2026-08-10 18:51:34.656 UTC [17945] LOG: shutting down
2026-08-10 18:51:34.783 UTC [17937] LOG: database system is shut down
2026-08-10T18:51:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50334 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:51:34 dist-test-slave-6zbq krb5kdc[20540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:51:34 dist-test-slave-6zbq krb5kdc[20540](info): set up 2 sockets
Aug 10 18:51:34 dist-test-slave-6zbq krb5kdc[20540](info): commencing operation
krb5kdc: starting...
W20260810 18:51:36.117341 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:51:36 dist-test-slave-6zbq krb5kdc[20540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:51:36.341529 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:32887
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:36.660869 20559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:36.661146 20559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:36.661276 20559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:36.668196 20559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:36.668303 20559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:36.668377 20559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:36.668452 20559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:36.668507 20559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:36.678013 20559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:32887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:36.679273 20559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:36.680513 20559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:36.691473 20566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:36.691517 20565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:36.691550 20569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:36.692539 20559 server_base.cc:1034] running on GCE node
I20260810 18:51:36.693221 20559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:36.694413 20559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:36.695608 20559 hybrid_clock.cc:648] HybridClock initialized: now 1786387896695561 us; error 39 us; skew 500 ppm
I20260810 18:51:36.695930 20559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:36 dist-test-slave-6zbq krb5kdc[20540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:36.698805 20559 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:36.700089 20559 webserver.cc:460] Webserver started at http://127.29.174.254:33961/ using document root <none> and password file <none>
I20260810 18:51:36.700688 20559 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:36.700793 20559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:36.701086 20559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:36.703928 20559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "248b1bda59794eda90324a12b84767db"
format_stamp: "Formatted at 2026-08-10 18:51:36 on dist-test-slave-6zbq"
server_key: "3dcbeb4cff7b747c0c9e1e1550272277"
server_key_iv: "78061e8b6d6d1f90e5b778155ef0303b"
server_key_version: "encryptionkey@0"
I20260810 18:51:36.704684 20559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "248b1bda59794eda90324a12b84767db"
format_stamp: "Formatted at 2026-08-10 18:51:36 on dist-test-slave-6zbq"
server_key: "3dcbeb4cff7b747c0c9e1e1550272277"
server_key_iv: "78061e8b6d6d1f90e5b778155ef0303b"
server_key_version: "encryptionkey@0"
I20260810 18:51:36.709638 20559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:51:36.713109 20582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:36.714444 20559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:36.714604 20559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "248b1bda59794eda90324a12b84767db"
format_stamp: "Formatted at 2026-08-10 18:51:36 on dist-test-slave-6zbq"
server_key: "3dcbeb4cff7b747c0c9e1e1550272277"
server_key_iv: "78061e8b6d6d1f90e5b778155ef0303b"
server_key_version: "encryptionkey@0"
I20260810 18:51:36.714775 20559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:36.728470 20559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:36.732686 20559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:36.732923 20559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:36.750226 20559 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:32887
I20260810 18:51:36.750272 20666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:32887 every 8 connection(s)
I20260810 18:51:36.751511 20559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:36.751848 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 20559
I20260810 18:51:36.752053 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:51:36.752947 30395 external_mini_cluster.cc:1428] Setting key 17e1c166d5515e5626b4343f7a0d085d
I20260810 18:51:36.755892 20667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:36.764127 20667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:51:36 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.174.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.253@KRBTEST.COM" created.
I20260810 18:51:36.787767 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.767306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:32887 (local address 127.0.0.1:35966)
0810 18:51:36.768181 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.768226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:36.770602 (+ 2376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:36.771406 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:36.771413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:36.772151 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:36.772599 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:36.772628 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:36.774105 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:36.774109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:36.775723 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:36.775738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:36.776650 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:36.776658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:36.776842 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:36.777480 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:36.777511 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:36.780186 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:36.783536 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:36.783546 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:36.783557 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:36.784095 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:36.784565 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:36.784569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:36.784572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:36.784740 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:36.785610 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:36.785616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:36.785808 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 18:51:36.785996 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 18:51:36.786400 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":370,"threads_started":1}
I20260810 18:51:36.787797 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.767306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.29.174.254:32887)
0810 18:51:36.769040 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:36.769046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:36.769087 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:51:36.770957 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:36.770962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:36.771176 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:36.771319 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:36.772793 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:36.773944 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:36.775850 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:36.776403 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:36.776532 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:36.780420 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:36.780429 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:36.780439 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:36.780467 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:36.783403 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:36.784236 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:36.784244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:36.784247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:36.784437 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:36.784855 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:36.784860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:36.784863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:36.785499 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:36.785703 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:36.786182 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 18:51:36.786561 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":1551,"spinlock_wait_cycles":14720,"thread_start_us":1318,"threads_started":1}
I20260810 18:51:36.790556 20667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:51:36.791536 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.791179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44638)
0810 18:51:36.791335 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.791436 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 18:51:36.791790 20585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
I20260810 18:51:36.793196 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.792868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44644)
0810 18:51:36.793007 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.793069 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:51:36.793390 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:32861: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
I20260810 18:51:36.811080 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:32861
--webserver_interface=127.29.174.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:36.842720 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } attempt: 1
I20260810 18:51:36.843744 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.843314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44658)
0810 18:51:36.843498 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.843599 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:51:36.844708 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.844405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44660)
0810 18:51:36.844518 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.844584 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:51:36.844889 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:32861: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
I20260810 18:51:36.913106 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } attempt: 2
I20260810 18:51:36.914328 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.913858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44672)
0810 18:51:36.914087 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.914177 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:51:36.915377 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:36.915040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44678)
0810 18:51:36.915155 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:36.915244 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 18:51:36.915572 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:32861: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
I20260810 18:51:37.006824 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } attempt: 3
I20260810 18:51:37.007947 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.007531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44692)
0810 18:51:37.007719 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.007811 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:51:37.008991 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.008656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44704)
0810 18:51:37.008776 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.008857 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:51:37.009187 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:32861: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
W20260810 18:51:37.125182 20679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:37.125430 20679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:37.125554 20679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:37.132390 20679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:37.132484 20679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:37.132537 20679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:37.132611 20679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:37.132674 20679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:37.141942 20679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.174.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:37.143115 20679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:37.144313 20679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:51:37.152412 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } attempt: 4
I20260810 18:51:37.153636 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.153020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44720)
0810 18:51:37.153359 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.153466 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260810 18:51:37.154826 20686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:37.155104 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.154521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44730)
0810 18:51:37.154784 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.154931 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260810 18:51:37.155357 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:32861: Network error: Client connection negotiation failed: client connection to 127.29.174.253:32861: connect: Connection refused (error 111)
W20260810 18:51:37.155498 20687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:37.156381 20679 server_base.cc:1034] running on GCE node
W20260810 18:51:37.156589 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:37.157356 20679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:37.158630 20679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:37.159837 20679 hybrid_clock.cc:648] HybridClock initialized: now 1786387897159791 us; error 31 us; skew 500 ppm
I20260810 18:51:37.160185 20679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:37.163131 20679 init.cc:377] Logged in from keytab as kudu/127.29.174.253@KRBTEST.COM (short username kudu)
I20260810 18:51:37.164465 20679 webserver.cc:460] Webserver started at http://127.29.174.253:36201/ using document root <none> and password file <none>
I20260810 18:51:37.165081 20679 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:37.165192 20679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:37.165474 20679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:37.168337 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data/instance:
uuid: "09ae061c484b473986f66498fb9be18d"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "25c411695588c16bc6c3550ea4f9fb96"
server_key_iv: "bc3747b365d278c5a62fbb3318f7dfc0"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.169126 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal/instance:
uuid: "09ae061c484b473986f66498fb9be18d"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "25c411695588c16bc6c3550ea4f9fb96"
server_key_iv: "bc3747b365d278c5a62fbb3318f7dfc0"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.173911 20679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260810 18:51:37.177088 20705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:37.178231 20679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:37.178401 20679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "09ae061c484b473986f66498fb9be18d"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "25c411695588c16bc6c3550ea4f9fb96"
server_key_iv: "bc3747b365d278c5a62fbb3318f7dfc0"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.178573 20679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:37.193882 20679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:37.197553 20679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:37.197788 20679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:37.214427 20679 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.253:32861
I20260810 18:51:37.214465 20799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.253:32861 every 8 connection(s)
I20260810 18:51:37.215622 20679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
I20260810 18:51:37.219129 20801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:37.221364 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 20679
I20260810 18:51:37.221524 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-1/wal/instance
I20260810 18:51:37.221938 30395 external_mini_cluster.cc:1428] Setting key 0fee3b437fa2eb41ece97f248ed3d1bc
I20260810 18:51:37.225325 20801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:37.245854 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.227756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.29.174.254:32887)
0810 18:51:37.227917 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.227921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.229127 (+ 1206us) server_negotiation.cc:407] Connection header received
0810 18:51:37.231333 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.231339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.231420 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.231519 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.233023 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.233689 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.235823 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.236270 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.236369 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.240129 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.240201 (+ 72us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.240218 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.240263 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.242871 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.243672 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.243675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.243677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.243792 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.244246 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.244251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.244254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.244562 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.244671 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.245467 (+ 796us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.245633 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:51:37.247313 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.228257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:32887 (local address 127.0.0.1:35968)
0810 18:51:37.228985 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.229033 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.231060 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.231721 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.231730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.232282 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.232856 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.232874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.233854 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.233860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.235687 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.235708 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.236484 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.236491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.236741 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.237373 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.237400 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.239885 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.243009 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.243017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.243032 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.243570 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.243903 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.243908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.243912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.244124 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.244694 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.244701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.245096 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.245395 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.245887 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":289,"threads_started":1}
WARNING: no policy specified for kudu/127.29.174.252@KRBTEST.COM; defaulting to no policy
I20260810 18:51:37.248875 20801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.252@KRBTEST.COM" created.
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:51:37.263163 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.249546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44746)
0810 18:51:37.249696 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.249714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.249792 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.250630 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.250650 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.251071 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.251234 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.251244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.252951 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.252954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.253747 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.253757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.254773 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.254778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.254963 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.255429 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.255445 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.257344 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.260668 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.260673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.260677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.261163 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.261494 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.261499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.261503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.261693 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.262215 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.262220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.262388 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.262532 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.262829 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":3}
I20260810 18:51:37.263221 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.249593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.29.174.253:32861)
0810 18:51:37.250156 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.250164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.250196 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:51:37.250268 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.250275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.250481 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.250590 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.251382 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.252830 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.253876 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.254650 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.254767 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.257556 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.257570 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.257585 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.257619 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.260525 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.261274 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.261280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.261282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.261389 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.261787 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.261790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.261792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.262084 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.262230 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.262644 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.262921 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
I20260810 18:51:37.265364 20801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:51:37.266413 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.265976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45922)
0810 18:51:37.266140 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.266252 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:51:37.266644 20714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
I20260810 18:51:37.268090 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.267772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45926)
0810 18:51:37.267932 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.268001 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:51:37.268286 20801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
I20260810 18:51:37.279100 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:36655
--webserver_interface=127.29.174.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:37065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:37.317541 20801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 1
I20260810 18:51:37.318459 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.318029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45938)
0810 18:51:37.318188 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.318285 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:51:37.319423 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.319097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45954)
0810 18:51:37.319258 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.319324 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 18:51:37.319595 20801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
I20260810 18:51:37.387779 20801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 2
I20260810 18:51:37.388833 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.388404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45964)
0810 18:51:37.388582 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.388685 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:51:37.389787 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.389442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45974)
0810 18:51:37.389585 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.389662 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:51:37.389992 20801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
I20260810 18:51:37.454604 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } attempt: 5
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:51:37.468019 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.455421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.29.174.253:32861)
0810 18:51:37.455582 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.455587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.455606 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:51:37.455820 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.455824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.455888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.455965 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.456969 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.457645 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.459383 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.459850 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.459944 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.463092 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.463097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.463100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.463141 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.465575 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.466186 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.466190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.466192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.466311 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.466717 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.466723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.466725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.466996 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.467141 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.467514 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.467716 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:51:37.468031 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.455280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44760)
0810 18:51:37.455516 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.455539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.455671 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.456031 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.456037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.456572 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.456820 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.456836 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.457788 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.457794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.459222 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.459241 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.460012 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.460019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.460225 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.460912 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.460931 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.462905 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.465719 (+ 2814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.465725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.465730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.466066 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.466439 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.466462 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.466466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.466603 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.467113 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.467118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.467264 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.467391 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.467696 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 18:51:37.469040 20667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:51:37.470026 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.469677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45982)
0810 18:51:37.469828 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.469902 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:51:37.470952 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.470635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:45998)
0810 18:51:37.470770 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.470848 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:51:37.471154 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
I20260810 18:51:37.481176 20801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 3
I20260810 18:51:37.482012 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.481625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46002)
0810 18:51:37.481765 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.481846 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:51:37.482985 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.482597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46018)
0810 18:51:37.482801 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.482877 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260810 18:51:37.483161 20801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
I20260810 18:51:37.522374 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 1
I20260810 18:51:37.523288 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.522884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46032)
0810 18:51:37.523057 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.523135 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 18:51:37.524209 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.523900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46034)
0810 18:51:37.524036 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.524109 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:51:37.524406 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
W20260810 18:51:37.592449 20813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260810 18:51:37.592602 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 2
W20260810 18:51:37.592731 20813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:37.592867 20813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:51:37.593640 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.593175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46036)
0810 18:51:37.593397 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.593483 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:51:37.594653 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.594302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46052)
0810 18:51:37.594460 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.594534 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 18:51:37.594833 20667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
W20260810 18:51:37.600095 20813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:37.600209 20813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:37.600283 20813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:37.600361 20813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:37.600422 20813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:37.610090 20813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:37065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:32887,127.29.174.253:32861,127.29.174.252:36655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:36655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.174.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:37.611398 20813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:37.612536 20813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:37.623191 20822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:37.623224 20827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:37.623248 20832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:37.624003 20813 server_base.cc:1034] running on GCE node
I20260810 18:51:37.624670 20813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:37.625854 20813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:37.626372 20801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 4
I20260810 18:51:37.627076 20813 hybrid_clock.cc:648] HybridClock initialized: now 1786387897627024 us; error 42 us; skew 500 ppm
I20260810 18:51:37.627350 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.626864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46058)
0810 18:51:37.627093 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.627193 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:51:37.627385 20813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:51:37.628252 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.627907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46064)
0810 18:51:37.628051 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.628127 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:51:37.628539 20801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:36655: Network error: Client connection negotiation failed: client connection to 127.29.174.252:36655: connect: Connection refused (error 111)
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:37.630378 20813 init.cc:377] Logged in from keytab as kudu/127.29.174.252@KRBTEST.COM (short username kudu)
I20260810 18:51:37.631650 20813 webserver.cc:460] Webserver started at http://127.29.174.252:40077/ using document root <none> and password file <none>
I20260810 18:51:37.632259 20813 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:37.632370 20813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:37.632632 20813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:37.635372 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data/instance:
uuid: "e999cd8bc1a84b02873bbcbe34a81d1e"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "3a2e8d9d915fff5ca5c09e269fc611df"
server_key_iv: "29908647223af68c98601235cd213100"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.636096 20813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal/instance:
uuid: "e999cd8bc1a84b02873bbcbe34a81d1e"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "3a2e8d9d915fff5ca5c09e269fc611df"
server_key_iv: "29908647223af68c98601235cd213100"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.640633 20813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:51:37.643700 20847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:37.644904 20813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:51:37.645051 20813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "e999cd8bc1a84b02873bbcbe34a81d1e"
format_stamp: "Formatted at 2026-08-10 18:51:37 on dist-test-slave-6zbq"
server_key: "3a2e8d9d915fff5ca5c09e269fc611df"
server_key_iv: "29908647223af68c98601235cd213100"
server_key_version: "encryptionkey@0"
I20260810 18:51:37.645222 20813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:37.659552 20813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:37.663300 20813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:37.663532 20813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:37.679951 20813 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.252:36655
I20260810 18:51:37.679989 20942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.252:36655 every 8 connection(s)
I20260810 18:51:37.681106 20813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
I20260810 18:51:37.684468 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:37.688175 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 20813
I20260810 18:51:37.688360 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387695213813-30395-0/minicluster-data/master-2/wal/instance
I20260810 18:51:37.688906 30395 external_mini_cluster.cc:1428] Setting key 1004a7b7bb75d5768feab40cb5ec3bf5
I20260810 18:51:37.691426 20943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:51:37.701012 20667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 3
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:51:37.711229 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.695043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.29.174.254:32887)
0810 18:51:37.695256 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.695263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.695815 (+ 552us) server_negotiation.cc:407] Connection header received
0810 18:51:37.695998 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.696004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.696066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.696146 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.697220 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.697863 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.699444 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.699925 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.700053 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.704288 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.704294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.704298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.704334 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.707721 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.708540 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.708546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.708549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.708733 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.709111 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.709117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.709121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.709553 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.709729 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.710472 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.710702 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:51:37.712491 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.690601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.29.174.252:36655)
0810 18:51:37.691297 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.691308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.691347 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:51:37.693300 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.693321 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.693683 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.693966 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.695632 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.696995 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.698690 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.699339 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.699498 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.704226 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.704238 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.704253 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.704289 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.707778 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.708609 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.708616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.708618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.708802 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.709277 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.709282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.709285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.709958 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.710239 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.711246 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.711492 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":507,"spinlock_wait_cycles":12672,"thread_start_us":311,"threads_started":1}
I20260810 18:51:37.712528 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.694996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:32887 (local address 127.0.0.1:35974)
0810 18:51:37.695692 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.695727 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.695879 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.696233 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.696240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.696789 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.697069 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.697080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.698007 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.698012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.699312 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.699325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.700131 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.700139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.700307 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.700795 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.700824 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.704089 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.707838 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.707850 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.707862 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.708404 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.708844 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.708850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.708853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.709004 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.709709 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.709718 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.710003 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.710377 (+ 374us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.710911 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":1,"thread_start_us":325,"threads_started":1}
I20260810 18:51:37.714005 20943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:51:37.717427 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.701488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46072)
0810 18:51:37.701754 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.701773 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.701873 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.702627 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.702631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.703021 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.703192 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.703204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.704694 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.704698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.705863 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.705876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.706839 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.706845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.707048 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.707643 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.707667 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.709975 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.714749 (+ 4774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.714754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.714757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.715237 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.715584 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.715588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.715591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.715761 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.716377 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.716382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.716585 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.716747 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.716996 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 18:51:37.717427 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.701534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.29.174.252:36655)
0810 18:51:37.702373 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.702378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.702399 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:51:37.702439 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.702442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.702489 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.702574 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.703372 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.704555 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.706001 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.706663 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.706813 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.710239 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.710246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.710251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.710282 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.713670 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.715345 (+ 1675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.715351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.715354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.715470 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.715894 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.715899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.715902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.716269 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.716397 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.716853 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.717054 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":567,"threads_started":1}
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:51:37.723544 20667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: Bootstrap starting.
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:37.727131 20667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:37.728569 20667 log.cc:824] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:37.730037 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.714674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:32861 (local address 127.0.0.1:44770)
0810 18:51:37.714837 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.714854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.715100 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.715577 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.715582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.715994 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.716200 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.716211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.717548 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.717553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.718890 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.718904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.719818 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.719824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.720046 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.720575 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.720595 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.723343 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.727367 (+ 4024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.727374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.727377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.727910 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.728362 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.728367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.728371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.728550 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.729248 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.729253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.729419 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.729536 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.729763 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:51:37.731199 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.714826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.29.174.253:32861)
0810 18:51:37.714982 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.714986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.715056 (+ 70us) server_negotiation.cc:407] Connection header received
0810 18:51:37.715213 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.715218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.715301 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.715389 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.716373 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.717360 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.719018 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.719661 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.719798 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.723552 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.723559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.723564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.723609 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.726955 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.728041 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.728046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.728049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.728231 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.728765 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.728770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.728774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.729099 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.730497 (+ 1398us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.730616 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.730849 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:51:37.731559 20667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: No bootstrap required, opened a new log
I20260810 18:51:37.731985 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.715893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.29.174.254:32887)
0810 18:51:37.716047 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.716052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.716168 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:51:37.716335 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.716340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.716406 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.716506 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.718015 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.719005 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.720475 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.720933 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.721040 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.725395 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.725406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.725410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.725446 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.728508 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.729295 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.729300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.729303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.729470 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.729909 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.729914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.729918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.730304 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.730610 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.731141 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.731665 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:51:37.732118 20943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:51:37.740453 20667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.740841 20667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:37.740958 20667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 248b1bda59794eda90324a12b84767db, State: Initialized, Role: FOLLOWER
I20260810 18:51:37.741720 20667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.742787 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:51:37.743290 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:37.744098 20667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:37.745787 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.732849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46088)
0810 18:51:37.732974 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.732990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.733083 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.733488 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.733492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.733944 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.734107 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.734119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.735538 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.735542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.736416 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.736429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.737118 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.737124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.737289 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.737857 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.737877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.740243 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.743308 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.743313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.743317 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.743722 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.744022 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.744027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.744031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.744151 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.744767 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.744772 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.744936 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.745100 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.745413 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 18:51:37.746948 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.732819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.29.174.252:36655)
0810 18:51:37.733226 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.733230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.733244 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:51:37.733283 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.733288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.733365 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.734236 (+ 871us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.734405 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.735403 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.736539 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.736982 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.737558 (+ 576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.740408 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.740414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.740417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.740442 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.743161 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.743824 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.743826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.743828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.743922 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.744261 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.744266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.744270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.744624 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.744777 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.745216 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.746703 (+ 1487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1238,"server-negotiator.queue_time_us":320}
I20260810 18:51:37.751893 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.733516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44784 (local address 127.29.174.253:32861)
0810 18:51:37.733664 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.733669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.733779 (+ 110us) server_negotiation.cc:407] Connection header received
0810 18:51:37.733916 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.733921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.733989 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.734087 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.735695 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.736767 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.738587 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.739192 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.739326 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.744492 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.744498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.744503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.744538 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.748622 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.749540 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.749546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.749549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.749707 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.750128 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.750133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.750137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.750526 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.750792 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.751343 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.751569 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:51:37.752360 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: Bootstrap starting.
I20260810 18:51:37.755599 20943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:37.756760 20943 log.cc:824] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:37.756810 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:51:37.759187 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: No bootstrap required, opened a new log
I20260810 18:51:37.767990 20943 raft_consensus.cc:348] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.768378 20943 raft_consensus.cc:374] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:37.768497 20943 raft_consensus.cc:729] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e999cd8bc1a84b02873bbcbe34a81d1e, State: Initialized, Role: FOLLOWER
I20260810 18:51:37.769249 20943 consensus_queue.cc:260] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.769414 20559 master_runner.cc:186] Error getting master registration for 127.29.174.253:32861: OK, instance_id { permanent_uuid: "09ae061c484b473986f66498fb9be18d" instance_seqno: 1786387897209729 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:51:37.780293 20559 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:51:37.798324 20982 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:37.798825 20982 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:37.799732 20943 sys_catalog.cc:564] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:51:37.801771 20977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:51:37.801971 20977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:51:37.813030 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.771818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.29.174.253:32861)
0810 18:51:37.771973 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.771978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.783325 (+ 11347us) server_negotiation.cc:407] Connection header received
0810 18:51:37.783560 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.783567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.783642 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.783708 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.790393 (+ 6685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.791529 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.796120 (+ 4591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.796746 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.796845 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.805346 (+ 8501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.805354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.805360 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.805409 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.809077 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.810455 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.810462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.810466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.810639 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.811106 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.811111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.811115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.811512 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.811743 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.812390 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.812644 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:51:37.813186 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.781191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.29.174.252:36655)
0810 18:51:37.781357 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.781363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.795957 (+ 14594us) server_negotiation.cc:407] Connection header received
0810 18:51:37.796098 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.796103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.796182 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.796266 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.798300 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.799357 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.801180 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.801896 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.801987 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.805435 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.805442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.805447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.805488 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.809078 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.810135 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.810141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.810145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.810312 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.810813 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.810818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.810821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.811182 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.811447 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.812615 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.812837 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:51:37.816294 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.766495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.29.174.254:32887)
0810 18:51:37.766677 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.766683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.780896 (+ 14213us) server_negotiation.cc:407] Connection header received
0810 18:51:37.781212 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.781217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.781312 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.781388 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.786917 (+ 5529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.788121 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.793597 (+ 5476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.794329 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.794456 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.809168 (+ 14712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.809176 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.809180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.809233 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.812704 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.813535 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.813541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.813544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.813737 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.814270 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.814277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.814281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.814719 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.815013 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.815593 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.815858 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 18:51:37.833091 21012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:51:37.833288 21012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:51:37.830248 20813 master_runner.cc:186] Error getting master registration for 127.29.174.253:32861: OK, instance_id { permanent_uuid: "09ae061c484b473986f66498fb9be18d" instance_seqno: 1786387897209729 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:51:37.840945 20813 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:51:37.841038 20982 raft_consensus.cc:482] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:37.841291 20982 raft_consensus.cc:504] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.842806 20982 leader_election.cc:288] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 248b1bda59794eda90324a12b84767db (127.29.174.254:32887), 09ae061c484b473986f66498fb9be18d (127.29.174.253:32861)
I20260810 18:51:37.843367 20631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248b1bda59794eda90324a12b84767db" is_pre_election: true
I20260810 18:51:37.843962 20631 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e999cd8bc1a84b02873bbcbe34a81d1e in term 0.
I20260810 18:51:37.844544 20852 leader_election.cc:302] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 248b1bda59794eda90324a12b84767db, e999cd8bc1a84b02873bbcbe34a81d1e; no voters:
I20260810 18:51:37.844944 20982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:37.845117 20982 raft_consensus.cc:482] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:37.845225 20982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:37.848819 20982 raft_consensus.cc:504] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.850404 20631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "248b1bda59794eda90324a12b84767db"
I20260810 18:51:37.850622 20631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:37.850908 20982 leader_election.cc:288] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [CANDIDATE]: Term 1 election: Requested vote from peers 248b1bda59794eda90324a12b84767db (127.29.174.254:32887), 09ae061c484b473986f66498fb9be18d (127.29.174.253:32861)
I20260810 18:51:37.854000 20631 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e999cd8bc1a84b02873bbcbe34a81d1e in term 1.
I20260810 18:51:37.854524 20852 leader_election.cc:302] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 248b1bda59794eda90324a12b84767db, e999cd8bc1a84b02873bbcbe34a81d1e; no voters:
I20260810 18:51:37.854909 20982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:37.855517 20982 raft_consensus.cc:686] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [term 1 LEADER]: Becoming Leader. State: Replica: e999cd8bc1a84b02873bbcbe34a81d1e, State: Running, Role: LEADER
I20260810 18:51:37.856192 20982 consensus_queue.cc:237] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:37.861523 21015 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e999cd8bc1a84b02873bbcbe34a81d1e. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:37.861924 21015 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:37.863495 21017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:37.867769 21017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:37.878042 20631 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [term 1 FOLLOWER]: Refusing update from remote peer e999cd8bc1a84b02873bbcbe34a81d1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:37.879048 20982 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:37.900104 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: SysCatalogTable state changed. Reason: New leader e999cd8bc1a84b02873bbcbe34a81d1e. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:37.900436 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:37.909739 21020 mvcc.cc:205] Tried to move back new op lower bound from 7317044829698498560 to 7317044829630615552. Current Snapshot: MvccSnapshot[applied={T|T < 7317044829698498560}]
I20260810 18:51:37.912886 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:37.913118 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:37.922444 20982 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:37.922777 20982 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:37.923552 21017 catalog_manager.cc:1347] Generated new cluster ID: e6530dee7f2d4d12bc551a36fe059b7c
I20260810 18:51:37.923655 21017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:37.927750 20801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } attempt: 5
Aug 10 18:51:37 dist-test-slave-6zbq krb5kdc[20540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:51:37.953889 20805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.928320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:36655 (local address 127.0.0.1:46102)
0810 18:51:37.928691 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:37.928712 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:37.928825 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:37.929701 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:37.929706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:37.930241 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:37.930497 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.930510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.932031 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.932037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.933450 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:37.933467 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.934826 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.934833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:37.935058 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.935801 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:37.935825 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:37.939272 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:37.943865 (+ 4593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.943870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.943874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.944969 (+ 1095us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.945960 (+ 991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:37.945970 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:37.945974 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:37.946304 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:37.947895 (+ 1591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:37.947901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:37.949170 (+ 1269us) client_negotiation.cc:769] Sending connection context
0810 18:51:37.949931 (+ 761us) client_negotiation.cc:240] Negotiation successful
0810 18:51:37.952241 (+ 2310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260810 18:51:37.953939 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:37.929027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.29.174.252:36655)
0810 18:51:37.929184 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:37.929190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:37.929207 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:51:37.929266 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:37.929271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:37.929344 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:37.929434 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:37.930680 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.931869 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.933621 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:37.934455 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:37.934575 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:37.939600 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:37.939607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:37.939613 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:37.939659 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:37.943579 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.945195 (+ 1616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.945201 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.945248 (+ 47us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.945627 (+ 379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:37.946542 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:37.946547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:37.946591 (+ 44us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:37.947533 (+ 942us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:37.949529 (+ 1996us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:37.950133 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 18:51:37.952189 (+ 2056us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:37.962198 20801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: Bootstrap starting.
I20260810 18:51:37.969528 20801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:37.970472 21017 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:51:37.972286 21017 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:37.972919 20801 log.cc:824] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:37.989221 20801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: No bootstrap required, opened a new log
I20260810 18:51:38.001297 21017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e: Generated new TSK 0
I20260810 18:51:38.003700 21017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:51:38.000766 20801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:38.004652 20801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:38.004828 20801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ae061c484b473986f66498fb9be18d, State: Initialized, Role: FOLLOWER
I20260810 18:51:38.005559 20801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } }
I20260810 18:51:38.011322 21029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:38.012326 20760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09ae061c484b473986f66498fb9be18d" is_pre_election: true
I20260810 18:51:38.012694 20752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:38.013633 20756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09ae061c484b473986f66498fb9be18d"
I20260810 18:51:38.014487 20801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:38.019505 20752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [term 1 FOLLOWER]: Refusing update from remote peer e999cd8bc1a84b02873bbcbe34a81d1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:38.020236 21015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:38.021124 21029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:38.088368 21029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e999cd8bc1a84b02873bbcbe34a81d1e. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:38.088689 21029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:38.089049 21042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:38.089490 21042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:51:38.137138 21058 catalog_manager.cc:1259] Loaded cluster ID: e6530dee7f2d4d12bc551a36fe059b7c
I20260810 18:51:38.137251 21058 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: loading cluster ID for follower catalog manager: success
I20260810 18:51:38.141005 21058 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: acquiring CA information for follower catalog manager: success
I20260810 18:51:38.143723 21058 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09ae061c484b473986f66498fb9be18d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:51:38.540625 21015 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "248b1bda59794eda90324a12b84767db" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 32887 } } peers { permanent_uuid: "09ae061c484b473986f66498fb9be18d" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 32861 } } peers { permanent_uuid: "e999cd8bc1a84b02873bbcbe34a81d1e" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 36655 } } }
I20260810 18:51:38.540895 21015 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999cd8bc1a84b02873bbcbe34a81d1e [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:38.803892 20977 catalog_manager.cc:1259] Loaded cluster ID: e6530dee7f2d4d12bc551a36fe059b7c
I20260810 18:51:38.803994 20977 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: loading cluster ID for follower catalog manager: success
I20260810 18:51:38.806330 20977 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: acquiring CA information for follower catalog manager: success
I20260810 18:51:38.808071 20977 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 248b1bda59794eda90324a12b84767db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:51:39.161608 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 20559
I20260810 18:51:39.162752 20661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:39.274474 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 20559
I20260810 18:51:39.300360 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 20679
I20260810 18:51:39.301499 20793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:39.405225 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 20679
I20260810 18:51:39.427593 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 20813
I20260810 18:51:39.428592 20935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:39.531390 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 20813
2026-08-10T18:51:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4644 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:51:39 dist-test-slave-6zbq krb5kdc[21076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:51:39 dist-test-slave-6zbq krb5kdc[21076](info): set up 2 sockets
Aug 10 18:51:39 dist-test-slave-6zbq krb5kdc[21076](info): commencing operation
krb5kdc: starting...
W20260810 18:51:40.753978 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:51:40 dist-test-slave-6zbq krb5kdc[21076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:51:40.977689 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38345
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:38345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:41.287597 21098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:41.287843 21098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:41.287963 21098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:41.294777 21098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:41.294862 21098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:41.294930 21098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:41.295002 21098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:41.295060 21098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:41.304167 21098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:38345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:41.305217 21098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:41.306370 21098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:41.316955 21104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:41.317018 21106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:41.317121 21111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:41.317798 21098 server_base.cc:1034] running on GCE node
I20260810 18:51:41.318471 21098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:41.319638 21098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:41.320834 21098 hybrid_clock.cc:648] HybridClock initialized: now 1786387901320786 us; error 31 us; skew 500 ppm
I20260810 18:51:41.321171 21098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:41 dist-test-slave-6zbq krb5kdc[21076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387901, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:41.324111 21098 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:41.325426 21098 webserver.cc:460] Webserver started at http://127.29.174.254:33323/ using document root <none> and password file <none>
I20260810 18:51:41.326035 21098 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:41.326148 21098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:41.326455 21098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:41.329118 21098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "fda2334ae2bb4084bdb4b43b672e96cc"
format_stamp: "Formatted at 2026-08-10 18:51:41 on dist-test-slave-6zbq"
server_key: "d7ab2372d806f136b5c381fef288911b"
server_key_iv: "2ef39d4b203e85146e3bdbffe048fb7b"
server_key_version: "encryptionkey@0"
I20260810 18:51:41.329810 21098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "fda2334ae2bb4084bdb4b43b672e96cc"
format_stamp: "Formatted at 2026-08-10 18:51:41 on dist-test-slave-6zbq"
server_key: "d7ab2372d806f136b5c381fef288911b"
server_key_iv: "2ef39d4b203e85146e3bdbffe048fb7b"
server_key_version: "encryptionkey@0"
I20260810 18:51:41.334455 21098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 18:51:41.337491 21120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:41.338636 21098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:41.338795 21098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "fda2334ae2bb4084bdb4b43b672e96cc"
format_stamp: "Formatted at 2026-08-10 18:51:41 on dist-test-slave-6zbq"
server_key: "d7ab2372d806f136b5c381fef288911b"
server_key_iv: "2ef39d4b203e85146e3bdbffe048fb7b"
server_key_version: "encryptionkey@0"
I20260810 18:51:41.338954 21098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:41.352257 21098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:41.355576 21098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:41.355816 21098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:41.372359 21098 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:38345
I20260810 18:51:41.372395 21200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:38345 every 8 connection(s)
I20260810 18:51:41.373517 21098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:41.375694 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 21098
I20260810 18:51:41.375921 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:51:41.376756 30395 external_mini_cluster.cc:1428] Setting key fd810958f22cdb1c9fe9abd4d8a2bb31
I20260810 18:51:41.377394 21202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:41.388716 21202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc: Bootstrap starting.
Aug 10 18:51:41 dist-test-slave-6zbq krb5kdc[21076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:41.391680 21202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:41.392665 21202 log.cc:824] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:41.395136 21202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc: No bootstrap required, opened a new log
I20260810 18:51:41.396544 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:41.378500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43674 (local address 127.29.174.254:38345)
0810 18:51:41.379199 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:41.379215 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:41.379252 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:51:41.380751 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:41.380771 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:41.381119 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:41.381456 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:41.382863 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.384168 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.385456 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:41.386120 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:41.386306 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:41.390091 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:41.390098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:41.390113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:41.390152 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:41.392937 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:41.393589 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:41.393595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:41.393600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:41.393753 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:41.394121 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:41.394124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:41.394126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:41.394634 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:41.394896 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:41.395338 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 18:51:41.395699 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":313,"threads_started":1}
I20260810 18:51:41.399881 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:51:41.402619 21202 raft_consensus.cc:348] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } }
I20260810 18:51:41.402981 21202 raft_consensus.cc:374] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:41.403066 21202 raft_consensus.cc:729] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda2334ae2bb4084bdb4b43b672e96cc, State: Initialized, Role: FOLLOWER
I20260810 18:51:41.403534 21202 consensus_queue.cc:260] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } }
I20260810 18:51:41.403745 21202 raft_consensus.cc:388] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:41.403862 21202 raft_consensus.cc:482] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:41.404013 21202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:41.405908 21202 raft_consensus.cc:504] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } }
I20260810 18:51:41.406333 21202 leader_election.cc:302] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fda2334ae2bb4084bdb4b43b672e96cc; no voters:
I20260810 18:51:41.406916 21202 leader_election.cc:288] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:51:41.407002 21208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:41.407622 21208 raft_consensus.cc:686] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [term 1 LEADER]: Becoming Leader. State: Replica: fda2334ae2bb4084bdb4b43b672e96cc, State: Running, Role: LEADER
I20260810 18:51:41.408138 21208 consensus_queue.cc:237] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } }
I20260810 18:51:41.408649 21202 sys_catalog.cc:564] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:41.410847 21209 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } } }
I20260810 18:51:41.411223 21209 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:41.411080 21210 sys_catalog.cc:455] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fda2334ae2bb4084bdb4b43b672e96cc. Latest consensus state: current_term: 1 leader_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fda2334ae2bb4084bdb4b43b672e96cc" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38345 } } }
I20260810 18:51:41.411346 21210 sys_catalog.cc:458] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:41.412900 21223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:41.417387 21223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:41.425509 21223 catalog_manager.cc:1347] Generated new cluster ID: ba04856305874279bddae8752985c195
I20260810 18:51:41.425609 21223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:41.441286 21223 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:51:41.442469 21223 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:41.455240 21223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fda2334ae2bb4084bdb4b43b672e96cc: Generated new TSK 0
I20260810 18:51:41.455906 21223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:51:41.510528 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 21098
I20260810 18:51:41.511680 21194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:41.605150 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 21098
2026-08-10T18:51:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2073 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:51:41 dist-test-slave-6zbq krb5kdc[21247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:51:41 dist-test-slave-6zbq krb5kdc[21247](info): set up 2 sockets
Aug 10 18:51:41 dist-test-slave-6zbq krb5kdc[21247](info): commencing operation
krb5kdc: starting...
W20260810 18:51:42.825924 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:51:42 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:51:43.057065 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:43.439604 21265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:43.439893 21265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:43.440027 21265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:43.447356 21265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:43.447474 21265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:43.447546 21265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:43.447620 21265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:43.447680 21265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:43.457072 21265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:43.458240 21265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:43.459515 21265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:43.470147 21273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:43.470279 21275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:43.470304 21278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:43.471143 21265 server_base.cc:1034] running on GCE node
I20260810 18:51:43.471845 21265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:43.473052 21265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:43.474251 21265 hybrid_clock.cc:648] HybridClock initialized: now 1786387903474210 us; error 46 us; skew 500 ppm
I20260810 18:51:43.474577 21265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:43 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387903, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:43.480640 21265 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:43.481942 21265 webserver.cc:460] Webserver started at http://127.29.174.254:38945/ using document root <none> and password file <none>
I20260810 18:51:43.482620 21265 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:43.482733 21265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:43.483006 21265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:43.485751 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "6e0eec2881c3421db2a74b15b9f0b59c"
format_stamp: "Formatted at 2026-08-10 18:51:43 on dist-test-slave-6zbq"
server_key: "3804717ca5a12646d1dff34b93021642"
server_key_iv: "db0724858649849a0961b61fd31f72b0"
server_key_version: "encryptionkey@0"
I20260810 18:51:43.486500 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "6e0eec2881c3421db2a74b15b9f0b59c"
format_stamp: "Formatted at 2026-08-10 18:51:43 on dist-test-slave-6zbq"
server_key: "3804717ca5a12646d1dff34b93021642"
server_key_iv: "db0724858649849a0961b61fd31f72b0"
server_key_version: "encryptionkey@0"
I20260810 18:51:43.491139 21265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:51:43.494275 21287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:43.495543 21265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:43.495704 21265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6e0eec2881c3421db2a74b15b9f0b59c"
format_stamp: "Formatted at 2026-08-10 18:51:43 on dist-test-slave-6zbq"
server_key: "3804717ca5a12646d1dff34b93021642"
server_key_iv: "db0724858649849a0961b61fd31f72b0"
server_key_version: "encryptionkey@0"
I20260810 18:51:43.495867 21265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:43.510741 21265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:43.514240 21265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:43.514499 21265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:43.531663 21265 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35015
I20260810 18:51:43.531694 21371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35015 every 8 connection(s)
I20260810 18:51:43.532855 21265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:43.534598 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 21265
I20260810 18:51:43.534809 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:51:43.535436 30395 external_mini_cluster.cc:1428] Setting key 122e5b568f8b0c6cfbf5d961b9283c68
I20260810 18:51:43.536474 21372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:51:43 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:43.549562 21372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap starting.
I20260810 18:51:43.553052 21372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:43.554332 21372 log.cc:824] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:43.557606 21372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: No bootstrap required, opened a new log
I20260810 18:51:43.559602 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:43.537268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.29.174.254:35015)
0810 18:51:43.538061 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:43.538076 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:43.538123 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:51:43.539850 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:43.539877 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:43.540345 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:43.540768 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:43.542017 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:43.544066 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:43.545399 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:43.546455 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:43.546642 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:43.550486 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:43.550497 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:43.550513 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:43.550553 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:43.554237 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:43.555189 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:43.555198 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:43.555206 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:43.555426 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:43.555946 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:43.555969 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:43.555972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:43.556604 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:43.556911 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:43.557548 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 18:51:43.558278 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":331,"threads_started":1}
I20260810 18:51:43.568562 21372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:43.568928 21372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:43.569025 21372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Initialized, Role: FOLLOWER
I20260810 18:51:43.569651 21372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:43.569921 21372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:43.570056 21372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:43.570238 21372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:43.572942 21372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:43.573516 21372 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0eec2881c3421db2a74b15b9f0b59c; no voters:
I20260810 18:51:43.574800 21372 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:51:43.574910 21382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:43.575546 21382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 LEADER]: Becoming Leader. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Running, Role: LEADER
I20260810 18:51:43.576117 21382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:43.576898 21372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:43.578835 21385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:43.578866 21387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0eec2881c3421db2a74b15b9f0b59c. Latest consensus state: current_term: 1 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:43.579274 21385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:43.579313 21387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:43.580248 21393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:43.588235 21393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:43.608085 21393 catalog_manager.cc:1347] Generated new cluster ID: 6556aa02713f46e8a0435e16caebfbaa
I20260810 18:51:43.608218 21393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:43.643530 21393 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:51:43.644884 21393 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:43.653991 21393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Generated new TSK 0
I20260810 18:51:43.654727 21393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:51:43.721542 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:44.050562 21410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:44.050815 21410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:44.050947 21410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:44.057760 21410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:44.057859 21410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:44.057997 21410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:51:44.067045 21410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:44.068087 21410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:44.069285 21410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:44.081735 21416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:44.081805 21418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:44.081779 21425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:44.082499 21410 server_base.cc:1034] running on GCE node
I20260810 18:51:44.083021 21410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:44.083830 21410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:44.085024 21410 hybrid_clock.cc:648] HybridClock initialized: now 1786387904084987 us; error 29 us; skew 500 ppm
I20260810 18:51:44.085315 21410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:44 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:44.088182 21410 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:51:44.089416 21410 webserver.cc:460] Webserver started at http://127.29.174.193:33657/ using document root <none> and password file <none>
I20260810 18:51:44.089984 21410 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:44.090101 21410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:44.090417 21410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:44.093039 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "4b9109d6505841d49c703c84bf56ca11"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "b2e3cc2cee2db6b67d3b56eec081011b"
server_key_iv: "9b0bf7614372df3da897c91203104e2a"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.093772 21410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "4b9109d6505841d49c703c84bf56ca11"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "b2e3cc2cee2db6b67d3b56eec081011b"
server_key_iv: "9b0bf7614372df3da897c91203104e2a"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.098574 21410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 18:51:44.101745 21435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.102914 21410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:51:44.103046 21410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "4b9109d6505841d49c703c84bf56ca11"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "b2e3cc2cee2db6b67d3b56eec081011b"
server_key_iv: "9b0bf7614372df3da897c91203104e2a"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.103220 21410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:44.124800 21410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:44.128278 21410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:44.128494 21410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:44.129288 21410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:44.130492 21410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:44.130565 21410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.130643 21410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:44.130685 21410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.159404 21410 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35419
I20260810 18:51:44.159448 21608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35419 every 8 connection(s)
I20260810 18:51:44.160537 21410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:44.166195 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 21410
I20260810 18:51:44.166412 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:51:44.167049 30395 external_mini_cluster.cc:1428] Setting key 98c9e606c4079c9c57117cc4eaab2b31
Aug 10 18:51:44 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:44.178869 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:44.162788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:60737 (local address 127.29.174.254:35015)
0810 18:51:44.163190 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:44.163194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:44.164021 (+ 827us) server_negotiation.cc:407] Connection header received
0810 18:51:44.165772 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:44.165778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:44.165889 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:44.165984 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:44.167474 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.168194 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.169702 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.170115 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.170227 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:44.173500 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:44.173505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:44.173508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:44.173539 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:44.175966 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:44.176647 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:44.176651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:44.176653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:44.176771 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:44.177188 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:44.177193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:44.177197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:44.177464 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:44.177615 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:44.178253 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 18:51:44.178507 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":217,"threads_started":1}
I20260810 18:51:44.179777 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:44.163224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:60737)
0810 18:51:44.163877 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:44.163919 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:44.165520 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:44.166226 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:44.166233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:44.166765 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:44.167313 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:44.167333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.168359 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.168363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:44.169554 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:44.169568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.170329 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.170336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:44.170574 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:44.171165 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:44.171187 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:44.173302 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:44.176101 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:44.176110 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:44.176125 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:44.176529 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:44.176873 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:44.176878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:44.176882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:44.177061 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:44.177590 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:44.177597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:44.177886 (+ 289us) client_negotiation.cc:769] Sending connection context
0810 18:51:44.178131 (+ 245us) client_negotiation.cc:240] Negotiation successful
0810 18:51:44.178644 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":250,"threads_started":1}
I20260810 18:51:44.181301 21609 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:44.181556 21609 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:44.182190 21609 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:44.184141 21315 ts_manager.cc:194] Registered new tserver with Master: 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419)
I20260810 18:51:44.185513 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:60737
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:51:44.222774 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:44.533802 21622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:44.534047 21622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:44.534157 21622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:44.541092 21622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:44.541188 21622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:44.541329 21622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:51:44.550441 21622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:44.551530 21622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:44.552749 21622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:44.564568 21636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:44.564654 21634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:44.564802 21630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:44.565531 21622 server_base.cc:1034] running on GCE node
I20260810 18:51:44.566116 21622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:44.566998 21622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:44.568189 21622 hybrid_clock.cc:648] HybridClock initialized: now 1786387904568138 us; error 44 us; skew 500 ppm
I20260810 18:51:44.568462 21622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:44 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:44.571339 21622 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:51:44.572542 21622 webserver.cc:460] Webserver started at http://127.29.174.194:36753/ using document root <none> and password file <none>
I20260810 18:51:44.573107 21622 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:44.573202 21622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:44.573421 21622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:44.576045 21622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "59d7114d0aa646afb5c4b6238176e006"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "c11c5d8d2d1a25732743799eaa45108a"
server_key_iv: "2a997a4c7693809124d6ab5bde7b42ff"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.576853 21622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "59d7114d0aa646afb5c4b6238176e006"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "c11c5d8d2d1a25732743799eaa45108a"
server_key_iv: "2a997a4c7693809124d6ab5bde7b42ff"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.581560 21622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:51:44.584595 21650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.585743 21622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:51:44.585868 21622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "59d7114d0aa646afb5c4b6238176e006"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "c11c5d8d2d1a25732743799eaa45108a"
server_key_iv: "2a997a4c7693809124d6ab5bde7b42ff"
server_key_version: "encryptionkey@0"
I20260810 18:51:44.586014 21622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:44.609449 21622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:44.613677 21622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:44.613916 21622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:44.614892 21622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:44.616039 21622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:44.616117 21622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.616227 21622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:44.616279 21622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:44.645179 21622 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:41107
I20260810 18:51:44.645236 21838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:41107 every 8 connection(s)
I20260810 18:51:44.646303 21622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:44.653492 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 21622
I20260810 18:51:44.653676 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:51:44.654192 30395 external_mini_cluster.cc:1428] Setting key eb3677a707300f590d6953b4806f3aa0
Aug 10 18:51:44 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387904, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:44.668910 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:44.648599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:50641 (local address 127.29.174.254:35015)
0810 18:51:44.648778 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:44.648783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:44.649845 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 18:51:44.651541 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:44.651546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:44.651613 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:44.651697 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:44.652968 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.653956 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.655671 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.656142 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.656272 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:44.663276 (+ 7004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:44.663283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:44.663287 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:44.663323 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:44.666071 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:44.666782 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:44.666788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:44.666791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:44.666966 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:44.667420 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:44.667425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:44.667428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:44.667768 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:44.667912 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:44.668504 (+ 592us) server_negotiation.cc:299] Negotiation successful
0810 18:51:44.668680 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:51:44.670035 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:44.649036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:50641)
0810 18:51:44.649698 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:44.649734 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:44.651321 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:44.651874 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:44.651880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:44.652322 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:44.652840 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:44.652862 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.654129 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.654134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:44.655525 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:44.655538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:44.656341 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:44.656345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:44.656524 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:44.660984 (+ 4460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:44.661006 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:44.663051 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:44.666190 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:44.666196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:44.666206 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:44.666658 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:44.667090 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:44.667095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:44.667098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:44.667306 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:44.667869 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:44.667874 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:44.668145 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 18:51:44.668383 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 18:51:44.668834 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":254,"threads_started":1}
I20260810 18:51:44.671345 21839 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:44.671615 21839 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:44.672271 21839 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:44.673535 21315 ts_manager.cc:194] Registered new tserver with Master: 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
I20260810 18:51:44.674667 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:50641
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:51:44.715000 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:45.034539 21850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:45.034814 21850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:45.034935 21850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:45.042882 21850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:45.042995 21850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:45.043150 21850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:51:45.052476 21850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:45.053643 21850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:45.054905 21850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:45.066998 21862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:45.067096 21861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:45.067066 21864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:45.067796 21850 server_base.cc:1034] running on GCE node
I20260810 18:51:45.068356 21850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:45.069188 21850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:45.070439 21850 hybrid_clock.cc:648] HybridClock initialized: now 1786387905070377 us; error 44 us; skew 500 ppm
I20260810 18:51:45.070755 21850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:45 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387905, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:45.073848 21850 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:51:45.075189 21850 webserver.cc:460] Webserver started at http://127.29.174.195:33597/ using document root <none> and password file <none>
I20260810 18:51:45.075812 21850 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:45.075927 21850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:45.076192 21850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:51:45.079115 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "03baadd652a4442cbdad36504e83c524"
format_stamp: "Formatted at 2026-08-10 18:51:45 on dist-test-slave-6zbq"
server_key: "529a034db65be866eca9b3564ca01dad"
server_key_iv: "7c7153034f685022320047241d5f77c7"
server_key_version: "encryptionkey@0"
I20260810 18:51:45.079852 21850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "03baadd652a4442cbdad36504e83c524"
format_stamp: "Formatted at 2026-08-10 18:51:45 on dist-test-slave-6zbq"
server_key: "529a034db65be866eca9b3564ca01dad"
server_key_iv: "7c7153034f685022320047241d5f77c7"
server_key_version: "encryptionkey@0"
I20260810 18:51:45.084640 21850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:51:45.087746 21874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:45.088961 21850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:51:45.089099 21850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "03baadd652a4442cbdad36504e83c524"
format_stamp: "Formatted at 2026-08-10 18:51:45 on dist-test-slave-6zbq"
server_key: "529a034db65be866eca9b3564ca01dad"
server_key_iv: "7c7153034f685022320047241d5f77c7"
server_key_version: "encryptionkey@0"
I20260810 18:51:45.089267 21850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:45.132892 21850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:45.136313 21850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:45.136554 21850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:45.137389 21850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:45.138567 21850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:45.138640 21850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:45.138765 21850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:45.138818 21850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:45.169781 21850 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34541
I20260810 18:51:45.169822 22072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34541 every 8 connection(s)
I20260810 18:51:45.170923 21850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:45.179364 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 21850
I20260810 18:51:45.179553 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:51:45.180114 30395 external_mini_cluster.cc:1428] Setting key 78b029679c71c24cc683997c668a3787
Aug 10 18:51:45 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387905, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:45.188154 21609 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:45.190364 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.173334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:37947 (local address 127.29.174.254:35015)
0810 18:51:45.173816 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.173823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.174687 (+ 864us) server_negotiation.cc:407] Connection header received
0810 18:51:45.176376 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.176381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.176447 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.176543 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:45.178010 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.178880 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.180956 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.181539 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.181651 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.185076 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:45.185081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:45.185085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:45.185116 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:45.187442 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.188076 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.188081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.188084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.188237 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.188703 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.188711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.188713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.189097 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:45.189247 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.189894 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.190074 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":211,"threads_started":1}
I20260810 18:51:45.191217 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.173826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:37947)
0810 18:51:45.174543 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.174579 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.176128 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.176718 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.176725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.177266 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:45.177860 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.177879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.179042 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.179047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.180809 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.180833 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.181772 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.181780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.182049 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.182734 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:45.182755 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:45.184891 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:45.187572 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:45.187579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:45.187588 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:45.187974 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:45.188381 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:45.188385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:45.188389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:45.188587 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:45.189215 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:45.189221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:45.189498 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.189785 (+ 287us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.190253 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":263,"threads_started":1}
I20260810 18:51:45.192437 22075 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:45.192667 22075 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:45.193331 22075 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:45.194761 21315 ts_manager.cc:194] Registered new tserver with Master: 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541)
I20260810 18:51:45.195520 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:45.195669 21315 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:37947
I20260810 18:51:45.212152 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.200399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.29.174.254:35015)
0810 18:51:45.200539 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.200545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.200933 (+ 388us) server_negotiation.cc:407] Connection header received
0810 18:51:45.201127 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.201132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.201199 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.201284 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:45.202625 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.203308 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.204596 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.204987 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.205089 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.207368 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:45.207373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:45.207376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:45.207404 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:45.209633 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.210262 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.210267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.210271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.210428 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.210761 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.210765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.210769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.211069 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:45.211318 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.211642 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.211851 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:51:45.216002 21315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:45.223031 21315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:45.244782 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.236765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.29.174.195:34541)
0810 18:51:45.237623 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.237631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.237660 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:51:45.237758 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.237764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.237966 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.238179 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:45.239151 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.240558 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.242808 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.243698 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.243873 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.244044 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.244286 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.244441 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":578,"threads_started":1}
I20260810 18:51:45.244887 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.236459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34541 (local address 127.0.0.1:56650)
0810 18:51:45.236945 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.236959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.237149 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.238528 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.238532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.238555 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:45.238958 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.238970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.240700 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.240709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.242615 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.242628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.243881 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.243887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.244062 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.244107 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.244230 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.244405 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":29,"spinlock_wait_cycles":1920,"thread_start_us":267,"threads_started":1}
I20260810 18:51:45.245755 22095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.234411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41107 (local address 127.0.0.1:34590)
0810 18:51:45.236946 (+ 2535us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.236966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.237113 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.237721 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.237728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.237760 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:45.238268 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.238287 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.240346 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.240352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.242654 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.242665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.244675 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.244682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.244987 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.245047 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.245126 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.245358 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"mutex_wait_us":161,"thread_start_us":1144,"threads_started":1}
I20260810 18:51:45.246141 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.234025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:59308)
0810 18:51:45.235061 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.235103 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.235263 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.237960 (+ 2697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.237963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.237984 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:45.238532 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.238546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.240045 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.240051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.242563 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.242586 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.244717 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.244724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.244983 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.245547 (+ 564us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.245646 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.245779 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":561,"thread_start_us":644,"threads_started":1}
I20260810 18:51:45.246620 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.234713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.29.174.194:41107)
0810 18:51:45.236615 (+ 1902us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.236623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.237114 (+ 491us) server_negotiation.cc:407] Connection header received
0810 18:51:45.237244 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.237253 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.237502 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.237698 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:45.238661 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.240212 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.242810 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.244172 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.244327 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.244494 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.245622 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.245782 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1762,"thread_start_us":1632,"threads_started":1}
I20260810 18:51:45.246980 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.234274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.29.174.193:35419)
0810 18:51:45.236218 (+ 1944us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.236229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.236263 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:51:45.237321 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.237329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.237500 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.237705 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:45.238769 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.239916 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.242763 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.244188 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.244358 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.244501 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.246469 (+ 1968us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.246625 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1692,"thread_start_us":1489,"threads_started":1}
I20260810 18:51:45.249424 21735 tablet_service.cc:1467] Processing CreateTablet for tablet 092dc135eea149e794a4732c9a8a5d99 (DEFAULT_TABLE table=test-table [id=454cc69ffcc54fbe85697caf459d0d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:45.249415 21960 tablet_service.cc:1467] Processing CreateTablet for tablet 092dc135eea149e794a4732c9a8a5d99 (DEFAULT_TABLE table=test-table [id=454cc69ffcc54fbe85697caf459d0d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:45.250669 21735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dc135eea149e794a4732c9a8a5d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:45.250299 21516 tablet_service.cc:1467] Processing CreateTablet for tablet 092dc135eea149e794a4732c9a8a5d99 (DEFAULT_TABLE table=test-table [id=454cc69ffcc54fbe85697caf459d0d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:45.250808 21960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dc135eea149e794a4732c9a8a5d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:45.251582 21516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 092dc135eea149e794a4732c9a8a5d99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:45.261147 22100 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Bootstrap starting.
I20260810 18:51:45.264611 22101 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Bootstrap starting.
I20260810 18:51:45.265388 22100 tablet_bootstrap.cc:654] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:45.266003 22103 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Bootstrap starting.
I20260810 18:51:45.266819 22100 log.cc:824] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:45.267855 22101 tablet_bootstrap.cc:654] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:45.268975 22101 log.cc:824] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:45.269014 22103 tablet_bootstrap.cc:654] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:45.269775 22100 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: No bootstrap required, opened a new log
I20260810 18:51:45.269999 22100 ts_tablet_manager.cc:1397] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:51:45.270105 22103 log.cc:824] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:45.271872 22101 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: No bootstrap required, opened a new log
I20260810 18:51:45.272146 22101 ts_tablet_manager.cc:1397] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260810 18:51:45.272330 22103 tablet_bootstrap.cc:492] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: No bootstrap required, opened a new log
I20260810 18:51:45.272571 22103 ts_tablet_manager.cc:1397] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:51:45.279803 22103 raft_consensus.cc:348] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.280135 22103 raft_consensus.cc:374] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:45.280241 22103 raft_consensus.cc:729] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9109d6505841d49c703c84bf56ca11, State: Initialized, Role: FOLLOWER
I20260810 18:51:45.280772 22103 consensus_queue.cc:260] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.280635 22100 raft_consensus.cc:348] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.281010 22100 raft_consensus.cc:374] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:45.281140 22100 raft_consensus.cc:729] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d7114d0aa646afb5c4b6238176e006, State: Initialized, Role: FOLLOWER
I20260810 18:51:45.281924 22100 consensus_queue.cc:260] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.282500 22103 ts_tablet_manager.cc:1428] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 18:51:45.282390 22101 raft_consensus.cc:348] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.282804 22101 raft_consensus.cc:374] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:45.282913 22101 raft_consensus.cc:729] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03baadd652a4442cbdad36504e83c524, State: Initialized, Role: FOLLOWER
I20260810 18:51:45.283491 22101 consensus_queue.cc:260] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.286242 21839 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:45.286418 22075 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:45.287868 22101 ts_tablet_manager.cc:1428] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Time spent starting tablet: real 0.016s user 0.012s sys 0.001s
I20260810 18:51:45.288182 22100 ts_tablet_manager.cc:1428] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Time spent starting tablet: real 0.018s user 0.010s sys 0.008s
W20260810 18:51:45.398878 21840 tablet.cc:2369] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:51:45.412588 21610 tablet.cc:2369] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:51:45.422366 22077 tablet.cc:2369] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:45.544329 22111 raft_consensus.cc:482] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:45.544587 22111 raft_consensus.cc:504] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.546216 22111 leader_election.cc:288] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
I20260810 18:51:45.552270 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.546414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.29.174.195:41021)
0810 18:51:45.546675 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.546694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.546812 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.547232 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.547237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.547266 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:45.547472 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.547484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.548679 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.548683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.550496 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.550513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.551766 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.551770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.551897 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.551921 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.551988 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.552077 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":1}
I20260810 18:51:45.552380 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.546662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:41021 (local address 127.29.174.193:35419)
0810 18:51:45.546865 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.546870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.546910 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:51:45.546993 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.546998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.547080 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.547192 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:45.547610 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.548549 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.550640 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.551626 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.551779 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.551851 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.552031 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.552147 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:51:45.553143 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.546667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41107 (local address 127.29.174.195:47641)
0810 18:51:45.547365 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:45.547381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:45.547497 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:45.547831 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:45.547835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:45.547856 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:45.548103 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.548111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.549194 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.549199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.550952 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:45.550963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.552643 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.552648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:45.552759 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.552782 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:51:45.552851 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 18:51:45.552929 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":369,"threads_started":1}
I20260810 18:51:45.553263 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.546798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:47641 (local address 127.29.174.194:41107)
0810 18:51:45.547001 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.547007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.547475 (+ 468us) server_negotiation.cc:407] Connection header received
0810 18:51:45.547616 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.547621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.547720 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.547806 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:45.548239 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.549045 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.551095 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.552198 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.552334 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.552401 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.552902 (+ 501us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.553026 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:51:45.553300 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dc135eea149e794a4732c9a8a5d99" candidate_uuid: "03baadd652a4442cbdad36504e83c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9109d6505841d49c703c84bf56ca11" is_pre_election: true
I20260810 18:51:45.553828 21545 raft_consensus.cc:2454] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03baadd652a4442cbdad36504e83c524 in term 0.
I20260810 18:51:45.554034 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dc135eea149e794a4732c9a8a5d99" candidate_uuid: "03baadd652a4442cbdad36504e83c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d7114d0aa646afb5c4b6238176e006" is_pre_election: true
I20260810 18:51:45.554451 21877 leader_election.cc:302] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03baadd652a4442cbdad36504e83c524, 4b9109d6505841d49c703c84bf56ca11; no voters:
I20260810 18:51:45.554490 21761 raft_consensus.cc:2454] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03baadd652a4442cbdad36504e83c524 in term 0.
I20260810 18:51:45.554808 22111 raft_consensus.cc:2781] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:45.554970 22111 raft_consensus.cc:482] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:45.555084 22111 raft_consensus.cc:3037] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:45.557212 22111 raft_consensus.cc:504] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.557717 22111 leader_election.cc:288] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [CANDIDATE]: Term 1 election: Requested vote from peers 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
I20260810 18:51:45.558116 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dc135eea149e794a4732c9a8a5d99" candidate_uuid: "03baadd652a4442cbdad36504e83c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9109d6505841d49c703c84bf56ca11"
I20260810 18:51:45.558151 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "092dc135eea149e794a4732c9a8a5d99" candidate_uuid: "03baadd652a4442cbdad36504e83c524" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59d7114d0aa646afb5c4b6238176e006"
I20260810 18:51:45.558305 21761 raft_consensus.cc:3037] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:45.558308 21545 raft_consensus.cc:3037] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:45.560438 21761 raft_consensus.cc:2454] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03baadd652a4442cbdad36504e83c524 in term 1.
I20260810 18:51:45.560617 21545 raft_consensus.cc:2454] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03baadd652a4442cbdad36504e83c524 in term 1.
I20260810 18:51:45.560797 21880 leader_election.cc:302] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006; no voters:
I20260810 18:51:45.561051 22111 raft_consensus.cc:2781] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:45.561475 22111 raft_consensus.cc:686] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 1 LEADER]: Becoming Leader. State: Replica: 03baadd652a4442cbdad36504e83c524, State: Running, Role: LEADER
I20260810 18:51:45.561945 22111 consensus_queue.cc:237] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } }
I20260810 18:51:45.565968 21315 catalog_manager.cc:5495] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03baadd652a4442cbdad36504e83c524 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "03baadd652a4442cbdad36504e83c524" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } health_report { overall_health: HEALTHY } } }
I20260810 18:51:45.606205 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.601572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56664 (local address 127.29.174.195:34541)
0810 18:51:45.601738 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.601743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.601785 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:51:45.601940 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.601944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.602014 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.602128 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:45.602422 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.603102 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.604426 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.604819 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.604905 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.605282 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:45.605744 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:45.605856 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.605893 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.605957 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:45.613204 21545 raft_consensus.cc:1261] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Refusing update from remote peer 03baadd652a4442cbdad36504e83c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:45.613210 21761 raft_consensus.cc:1261] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Refusing update from remote peer 03baadd652a4442cbdad36504e83c524: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:45.614068 22111 consensus_queue.cc:1035] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:45.614367 22115 consensus_queue.cc:1035] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:45.624712 22127 mvcc.cc:205] Tried to move back new op lower bound from 7317044861383376896 to 7317044861185277952. Current Snapshot: MvccSnapshot[applied={T|T < 7317044861383376896}]
I20260810 18:51:45.637741 21315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:45.637949 21315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:45.641288 21315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Sending DeleteTablet for 3 replicas of tablet 092dc135eea149e794a4732c9a8a5d99
I20260810 18:51:45.642261 21735 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dc135eea149e794a4732c9a8a5d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:45 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:34590
I20260810 18:51:45.642261 21516 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dc135eea149e794a4732c9a8a5d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:45 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59308
I20260810 18:51:45.642495 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 21410
I20260810 18:51:45.642623 21960 tablet_service.cc:1514] Processing DeleteTablet for tablet 092dc135eea149e794a4732c9a8a5d99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:45 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:56650
I20260810 18:51:45.667943 22143 tablet_replica.cc:330] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: stopping tablet replica
I20260810 18:51:45.668355 22143 raft_consensus.cc:2229] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:45.668840 22143 raft_consensus.cc:2258] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:45.669514 22141 tablet_replica.cc:330] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: stopping tablet replica
I20260810 18:51:45.669893 22141 raft_consensus.cc:2229] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:45.670357 22141 raft_consensus.cc:2258] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:45.672518 22141 ts_tablet_manager.cc:1905] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:45.672662 22143 ts_tablet_manager.cc:1905] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:45.677572 22147 tablet_replica.cc:330] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: stopping tablet replica
I20260810 18:51:45.677942 22147 raft_consensus.cc:2229] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:51:45.678646 22147 raft_consensus.cc:2258] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:45.679750 22143 ts_tablet_manager.cc:1918] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:45.679888 22143 log.cc:1196] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/092dc135eea149e794a4732c9a8a5d99
I20260810 18:51:45.680299 22143 ts_tablet_manager.cc:1939] T 092dc135eea149e794a4732c9a8a5d99 P 59d7114d0aa646afb5c4b6238176e006: Deleting consensus metadata
I20260810 18:51:45.681746 21294 catalog_manager.cc:4841] TS 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): tablet 092dc135eea149e794a4732c9a8a5d99 (table test-table [id=454cc69ffcc54fbe85697caf459d0d8b]) successfully deleted
I20260810 18:51:45.682657 22147 ts_tablet_manager.cc:1905] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:45.683866 22141 ts_tablet_manager.cc:1918] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:45.684021 22141 log.cc:1196] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/092dc135eea149e794a4732c9a8a5d99
I20260810 18:51:45.684513 22141 ts_tablet_manager.cc:1939] T 092dc135eea149e794a4732c9a8a5d99 P 4b9109d6505841d49c703c84bf56ca11: Deleting consensus metadata
I20260810 18:51:45.686193 21290 catalog_manager.cc:4841] TS 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419): tablet 092dc135eea149e794a4732c9a8a5d99 (table test-table [id=454cc69ffcc54fbe85697caf459d0d8b]) successfully deleted
I20260810 18:51:45.687734 22147 ts_tablet_manager.cc:1918] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:45.687868 22147 log.cc:1196] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/092dc135eea149e794a4732c9a8a5d99
I20260810 18:51:45.688282 22147 ts_tablet_manager.cc:1939] T 092dc135eea149e794a4732c9a8a5d99 P 03baadd652a4442cbdad36504e83c524: Deleting consensus metadata
Aug 10 18:51:45 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:51:45.689678 21288 catalog_manager.cc:4841] TS 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): tablet 092dc135eea149e794a4732c9a8a5d99 (table test-table [id=454cc69ffcc54fbe85697caf459d0d8b]) successfully deleted
I20260810 18:51:45.696219 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.643748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59310 (local address 127.29.174.193:35419)
0810 18:51:45.643921 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.643927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.663116 (+ 19189us) server_negotiation.cc:407] Connection header received
0810 18:51:45.663805 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.663811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.663886 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.663951 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:45.670537 (+ 6586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.671629 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.673661 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.674277 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.674391 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.690576 (+ 16185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:45.690587 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:45.690600 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:45.690638 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:45.693314 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.694122 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.694128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.694131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.694272 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.694658 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.694662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.694664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.695013 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:45.695293 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.695629 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.695867 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:51:45.696803 21599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:45.843977 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 21410
I20260810 18:51:45.872834 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 21622
Aug 10 18:51:45 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:51:45.888059 22096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.873608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.29.174.194:41107)
0810 18:51:45.873793 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.873799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.873820 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:51:45.873932 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.873937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.874019 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.874125 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:45.875454 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.876519 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.877888 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.878547 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.878682 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:45.882917 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:45.882930 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:45.882944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:45.882975 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:45.885551 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.886267 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.886273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.886275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.886421 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:45.886774 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:45.886776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:45.886778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:45.887125 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:45.887342 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:45.887630 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 18:51:45.887779 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:51:45.888763 21833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:45.975083 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 21622
I20260810 18:51:45.994068 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 21850
Aug 10 18:51:45 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:51:46.010556 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:45.994877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.29.174.195:34541)
0810 18:51:45.995070 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:45.995076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:45.995099 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:51:45.995212 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:45.995218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:45.995300 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:45.995413 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:45.996669 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:45.997976 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:45.999639 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:46.000308 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:46.000454 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:46.004812 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:46.004823 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:46.004839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:46.004878 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:46.007752 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:46.008545 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:46.008551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:46.008554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:46.008691 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:46.009089 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:46.009092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:46.009094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:46.009445 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:46.009662 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:46.010037 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 18:51:46.010195 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:51:46.011282 22063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:46.129770 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 21850
I20260810 18:51:46.147369 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 21265
I20260810 18:51:46.148386 21365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:46.253804 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 21265
I20260810 18:51:46.277693 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--webserver_interface=127.29.174.254
--webserver_port=38945
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:46.598538 22159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:46.598825 22159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:46.598963 22159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:46.607478 22159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:46.607595 22159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:46.607687 22159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:46.607774 22159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:46.607832 22159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:46.618638 22159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=38945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:46.619921 22159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:46.621243 22159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:46.647691 22168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:46.647809 22169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:46.648025 22173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:46.648957 22159 server_base.cc:1034] running on GCE node
I20260810 18:51:46.649832 22159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:46.651460 22159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:46.652706 22159 hybrid_clock.cc:648] HybridClock initialized: now 1786387906652654 us; error 39 us; skew 500 ppm
I20260810 18:51:46.653101 22159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:46 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387906, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:46.656875 22159 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:46.658558 22159 webserver.cc:460] Webserver started at http://127.29.174.254:38945/ using document root <none> and password file <none>
I20260810 18:51:46.659308 22159 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:46.659451 22159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:46.666133 22159 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.007s
I20260810 18:51:46.669009 22183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:46.670377 22159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:51:46.670555 22159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6e0eec2881c3421db2a74b15b9f0b59c"
format_stamp: "Formatted at 2026-08-10 18:51:43 on dist-test-slave-6zbq"
server_key: "3804717ca5a12646d1dff34b93021642"
server_key_iv: "db0724858649849a0961b61fd31f72b0"
server_key_version: "encryptionkey@0"
I20260810 18:51:46.671602 22159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:46.695681 22159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:46.699841 22159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:46.700096 22159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:46.717495 22159 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35015
I20260810 18:51:46.717519 22286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35015 every 8 connection(s)
I20260810 18:51:46.718919 22159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:46.721791 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 22159
I20260810 18:51:46.722812 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:35419
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=33657
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:46.727501 22288 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:46.731849 22288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap starting.
I20260810 18:51:46.751504 22288 log.cc:824] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:46.763495 22288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:46.763981 22288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap complete.
I20260810 18:51:46.772994 22288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:46.773778 22288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Initialized, Role: FOLLOWER
I20260810 18:51:46.774435 22288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:46.774686 22288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:46.774822 22288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:46.774983 22288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:51:46.778791 22288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:46.779291 22288 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0eec2881c3421db2a74b15b9f0b59c; no voters:
I20260810 18:51:46.779899 22288 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:51:46.780038 22292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:51:46.781129 22292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 LEADER]: Becoming Leader. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Running, Role: LEADER
I20260810 18:51:46.781649 22292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:46.782051 22288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:46.784212 22296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0eec2881c3421db2a74b15b9f0b59c. Latest consensus state: current_term: 2 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:46.784603 22296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:46.784360 22295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:46.784729 22295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:46.795708 22306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:46.804318 22306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:46.805775 22306 catalog_manager.cc:1259] Loaded cluster ID: 6556aa02713f46e8a0435e16caebfbaa
I20260810 18:51:46.805876 22306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:51:46.809252 22306 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 18:51:46.827945 22306 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 18:51:46.836406 22306 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:51:46.837581 22306 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Loaded TSK: 0
I20260810 18:51:46.838701 22306 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:51:47.082996 22290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:47.083274 22290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:47.083412 22290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:47.090430 22290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:47.090516 22290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:47.090644 22290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:51:47.100095 22290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:35419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=33657
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:47.101213 22290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:47.102454 22290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:47.116483 22331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.116822 22344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.116396 22338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:47.117817 22290 server_base.cc:1034] running on GCE node
I20260810 18:51:47.118412 22290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:47.119315 22290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:47.120528 22290 hybrid_clock.cc:648] HybridClock initialized: now 1786387907120480 us; error 44 us; skew 500 ppm
I20260810 18:51:47.120812 22290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:47 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:47.123919 22290 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:51:47.125305 22290 webserver.cc:460] Webserver started at http://127.29.174.193:33657/ using document root <none> and password file <none>
I20260810 18:51:47.125897 22290 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:47.126026 22290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:47.130990 22290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:51:47.133445 22359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.134723 22290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:47.134861 22290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "4b9109d6505841d49c703c84bf56ca11"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "b2e3cc2cee2db6b67d3b56eec081011b"
server_key_iv: "9b0bf7614372df3da897c91203104e2a"
server_key_version: "encryptionkey@0"
I20260810 18:51:47.135761 22290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:47.159279 22290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:47.163149 22290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:47.163393 22290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:47.164381 22290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:47.165737 22290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:47.165822 22290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.165923 22290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:47.165993 22290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.196823 22290 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35419
I20260810 18:51:47.196885 22538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35419 every 8 connection(s)
I20260810 18:51:47.198179 22290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:47.201359 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 22290
I20260810 18:51:47.202139 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:41107
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=36753
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:47 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:47.219836 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:47.200792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:42343 (local address 127.29.174.254:35015)
0810 18:51:47.201363 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:47.201375 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:47.202045 (+ 670us) server_negotiation.cc:407] Connection header received
0810 18:51:47.204270 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:47.204282 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:47.204572 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:47.204882 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:47.206408 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.207616 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.209071 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.209700 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.209868 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:47.213109 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:47.213121 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:47.213134 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:47.213190 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:47.215894 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:47.216707 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:47.216713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:47.216718 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:47.216866 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:47.217337 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:47.217340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:47.217342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:47.217740 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:47.217877 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:47.218709 (+ 832us) server_negotiation.cc:299] Negotiation successful
0810 18:51:47.218987 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
I20260810 18:51:47.220170 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:47.201104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:42343)
0810 18:51:47.201881 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:47.201924 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:47.203781 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:47.205052 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:47.205063 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:47.205622 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:47.206152 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:47.206171 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.207765 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.207769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:47.208934 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:47.208946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.209929 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.209937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:47.210184 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:47.210789 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:47.210814 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:47.212841 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:47.216031 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:47.216040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:47.216052 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:47.216580 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:47.216967 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:47.216972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:47.216975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:47.217235 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:47.217868 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:47.217875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:47.218261 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 18:51:47.218563 (+ 302us) client_negotiation.cc:240] Negotiation successful
0810 18:51:47.219027 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":324,"threads_started":1}
I20260810 18:51:47.222703 22542 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:47.222967 22542 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:47.223644 22542 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:47.225679 22212 ts_manager.cc:194] Registered new tserver with Master: 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419)
I20260810 18:51:47.227156 22212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:42343
W20260810 18:51:47.524678 22548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:47.524920 22548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:47.525038 22548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:47.532176 22548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:47.532271 22548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:47.532413 22548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:51:47.541659 22548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:41107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=36753
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:47.542806 22548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:47.544030 22548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:47.556288 22558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.556260 22559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.556320 22561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:47.558094 22548 server_base.cc:1034] running on GCE node
I20260810 18:51:47.558772 22548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:47.559712 22548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:47.560905 22548 hybrid_clock.cc:648] HybridClock initialized: now 1786387907560858 us; error 44 us; skew 500 ppm
I20260810 18:51:47.561203 22548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:47 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:47.564070 22548 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:51:47.565395 22548 webserver.cc:460] Webserver started at http://127.29.174.194:36753/ using document root <none> and password file <none>
I20260810 18:51:47.566071 22548 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:47.566205 22548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:47.571244 22548 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:51:47.573767 22569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.575001 22548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:47.575135 22548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "59d7114d0aa646afb5c4b6238176e006"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "c11c5d8d2d1a25732743799eaa45108a"
server_key_iv: "2a997a4c7693809124d6ab5bde7b42ff"
server_key_version: "encryptionkey@0"
I20260810 18:51:47.576119 22548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:47.592206 22548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:47.595646 22548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:47.595872 22548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:47.596761 22548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:47.597990 22548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:47.598088 22548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.598191 22548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:47.598258 22548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:47.628511 22548 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:41107
I20260810 18:51:47.628574 22767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:41107 every 8 connection(s)
I20260810 18:51:47.629882 22548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:47.639952 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 22548
I20260810 18:51:47.640681 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:34541
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=33597
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:51:47 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:47.648607 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:47.632155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:59997 (local address 127.29.174.254:35015)
0810 18:51:47.632321 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:47.632326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:47.633494 (+ 1168us) server_negotiation.cc:407] Connection header received
0810 18:51:47.635284 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:47.635289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:47.635359 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:47.635446 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:47.636737 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.637395 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.638874 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.639329 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.639420 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:47.643197 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:47.643204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:47.643209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:47.643250 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:47.645603 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:47.646440 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:47.646445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:47.646448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:47.646586 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:47.646998 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:47.647001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:47.647004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:47.647295 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:47.647410 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:47.648106 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 18:51:47.648313 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:51:47.649631 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:47.632551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:59997)
0810 18:51:47.633331 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:47.633376 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:47.635037 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:47.635634 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:47.635641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:47.636088 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:47.636613 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:47.636629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.637515 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.637519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:47.638741 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:47.638754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:47.639531 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:47.639539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:47.639837 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:47.640467 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:47.640495 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:47.642985 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:47.645751 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:47.645759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:47.645773 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:47.646292 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:47.646684 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:47.646688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:47.646691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:47.646904 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:47.647407 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:47.647414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:47.647765 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 18:51:47.648029 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 18:51:47.648483 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":323,"threads_started":1}
I20260810 18:51:47.650933 22770 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:47.651243 22770 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:47.651984 22770 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:47.653323 22212 ts_manager.cc:194] Registered new tserver with Master: 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
I20260810 18:51:47.654389 22212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:59997
W20260810 18:51:47.962208 22778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:47.962512 22778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:47.962621 22778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:47.969616 22778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:47.969713 22778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:47.969821 22778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:51:47.979393 22778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:34541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=33597
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:47.980453 22778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:47.981649 22778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:47.993614 22790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.993695 22784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:47.993674 22786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:47.994408 22778 server_base.cc:1034] running on GCE node
I20260810 18:51:47.994969 22778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:47.995939 22778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:47.997155 22778 hybrid_clock.cc:648] HybridClock initialized: now 1786387907997099 us; error 63 us; skew 500 ppm
I20260810 18:51:47.997460 22778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:47 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:48.000464 22778 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:51:48.001732 22778 webserver.cc:460] Webserver started at http://127.29.174.195:33597/ using document root <none> and password file <none>
I20260810 18:51:48.002331 22778 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:48.002492 22778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:48.007274 22778 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:51:48.009651 22799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:48.010849 22778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:51:48.010987 22778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "03baadd652a4442cbdad36504e83c524"
format_stamp: "Formatted at 2026-08-10 18:51:45 on dist-test-slave-6zbq"
server_key: "529a034db65be866eca9b3564ca01dad"
server_key_iv: "7c7153034f685022320047241d5f77c7"
server_key_version: "encryptionkey@0"
I20260810 18:51:48.011873 22778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:48.028354 22778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:48.031754 22778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:48.031973 22778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:48.032838 22778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:48.034017 22778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:48.034099 22778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:48.034198 22778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:48.034271 22778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:48.063627 22778 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34541
I20260810 18:51:48.063700 22990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34541 every 8 connection(s)
I20260810 18:51:48.064918 22778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:48.067523 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 22778
Aug 10 18:51:48 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387907, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:51:48.084913 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.067154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:57643 (local address 127.29.174.254:35015)
0810 18:51:48.067360 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.067366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.068546 (+ 1180us) server_negotiation.cc:407] Connection header received
0810 18:51:48.070326 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.070332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.070447 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.070541 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:48.072017 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.072729 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.074918 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.075367 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.075497 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.079211 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:48.079218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:48.079222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:48.079275 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:48.081941 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.082619 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.082624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.082628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.082778 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.083245 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.083249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.083251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.083526 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:48.083682 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.084319 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.084601 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:51:48.085644 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.067622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:57643)
0810 18:51:48.068388 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.068439 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.070068 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.070897 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.070906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.071356 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:48.071851 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.071867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.072886 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.072893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.074771 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.074787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.075610 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.075618 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.075861 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.076411 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:48.076431 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:48.079010 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:48.082061 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:48.082067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:48.082077 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:48.082510 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:48.082901 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:48.082905 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:48.082909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:48.083121 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:48.083641 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:48.083648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:48.083958 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.084167 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.084619 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":304,"threads_started":1}
I20260810 18:51:48.086943 22993 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35015
I20260810 18:51:48.087240 22993 heartbeater.cc:464] Registering TS with master...
I20260810 18:51:48.087299 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.068343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.29.174.254:35015)
0810 18:51:48.073153 (+ 4810us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.073159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.073663 (+ 504us) server_negotiation.cc:407] Connection header received
0810 18:51:48.073829 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.073833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.073899 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.073989 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:48.075715 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.076509 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.078131 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.078781 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.078914 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.081814 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:48.081822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:48.081826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:48.081882 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:48.084625 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.085321 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.085326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.085330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.085459 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.085862 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.085868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.085871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.086192 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:48.086457 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.086764 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.086963 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4704,"thread_start_us":4588,"threads_started":1}
I20260810 18:51:48.088025 22993 heartbeater.cc:511] Master 127.29.174.254:35015 requested a full tablet report, sending...
I20260810 18:51:48.089534 22211 ts_manager.cc:194] Registered new tserver with Master: 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541)
I20260810 18:51:48.090385 22211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:57643
I20260810 18:51:48.101289 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:51:48.107300 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.102263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45982 (local address 127.29.174.254:35015)
0810 18:51:48.102458 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.102463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.102818 (+ 355us) server_negotiation.cc:407] Connection header received
0810 18:51:48.102973 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.102976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.103091 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.103201 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:48.103530 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.104175 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.105525 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.105932 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.106016 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.106457 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:48.106889 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:48.106996 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.107075 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.107131 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:51:48.108042 22211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:51:48.114907 22211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:51:48.150766 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.125292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34541 (local address 127.0.0.1:34954)
0810 18:51:48.134889 (+ 9597us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.134923 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.135087 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.144093 (+ 9006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.144098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.144122 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:48.144516 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.144530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.146297 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.146302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.147975 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.147991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.149681 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.149688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.149891 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.149943 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.150099 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.150278 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9363,"mutex_wait_us":48,"thread_start_us":15946,"threads_started":1}
I20260810 18:51:48.151124 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.125537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.29.174.195:34541)
0810 18:51:48.143508 (+ 17971us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.143515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.143544 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:51:48.143630 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.143637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.143838 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.144074 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:48.144730 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.146174 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.148155 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.149477 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.149663 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.149842 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.150148 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.150623 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17819,"thread_start_us":17676,"threads_started":1}
I20260810 18:51:48.151494 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.125767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41107 (local address 127.0.0.1:44574)
0810 18:51:48.141555 (+ 15788us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.141579 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.141722 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.143658 (+ 1936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.143662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.143687 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:48.144105 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.144117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.146017 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.146023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.148395 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.148410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.150535 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.150542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.150855 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.150889 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.150970 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.151086 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15621,"mutex_wait_us":125,"thread_start_us":16280,"threads_started":1}
I20260810 18:51:48.151548 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.126062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34238)
0810 18:51:48.141842 (+ 15780us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.141860 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.141973 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.143480 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.143488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.143519 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:48.144004 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.144019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.145790 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.145795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.148750 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.148765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.150553 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.150559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.150789 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.150960 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.151044 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.151162 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15645,"mutex_wait_us":91,"thread_start_us":8479,"threads_started":1}
I20260810 18:51:48.152235 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.126645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.29.174.193:35419)
0810 18:51:48.142775 (+ 16130us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.142785 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.142816 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:51:48.142917 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.142925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.143177 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.143371 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:48.144247 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.145648 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.148942 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.150033 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.150177 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.150376 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.151763 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.151925 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15969,"thread_start_us":15865,"threads_started":1}
I20260810 18:51:48.152416 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.126255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.29.174.194:41107)
0810 18:51:48.143118 (+ 16863us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.143126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.143159 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:51:48.143245 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.143264 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.143473 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.143652 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:48.144348 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.145876 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.148592 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.150018 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.150191 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.150674 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.151783 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.151940 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16704,"thread_start_us":16579,"threads_started":1}
I20260810 18:51:48.155125 22430 tablet_service.cc:1467] Processing CreateTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 (DEFAULT_TABLE table=test-table [id=d3716771fea8422fb0201380588a8010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:48.155170 22879 tablet_service.cc:1467] Processing CreateTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 (DEFAULT_TABLE table=test-table [id=d3716771fea8422fb0201380588a8010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:48.155463 22660 tablet_service.cc:1467] Processing CreateTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 (DEFAULT_TABLE table=test-table [id=d3716771fea8422fb0201380588a8010]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:51:48.156327 22430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95be66a0ec3c4d2aa32cc80b0bde0512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:48.156504 22660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95be66a0ec3c4d2aa32cc80b0bde0512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:48.156505 22879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95be66a0ec3c4d2aa32cc80b0bde0512. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:51:48.167608 23016 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Bootstrap starting.
I20260810 18:51:48.170514 23016 tablet_bootstrap.cc:654] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:48.171651 23016 log.cc:824] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:48.172537 23020 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Bootstrap starting.
I20260810 18:51:48.174096 23019 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Bootstrap starting.
I20260810 18:51:48.175709 23016 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: No bootstrap required, opened a new log
I20260810 18:51:48.175926 23020 tablet_bootstrap.cc:654] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:48.175954 23016 ts_tablet_manager.cc:1397] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 18:51:48.177134 23020 log.cc:824] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:48.177624 23019 tablet_bootstrap.cc:654] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Neither blocks nor log segments found. Creating new log.
I20260810 18:51:48.178745 23019 log.cc:824] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:48.180094 23020 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: No bootstrap required, opened a new log
I20260810 18:51:48.180363 23020 ts_tablet_manager.cc:1397] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:51:48.183801 23016 raft_consensus.cc:348] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.184118 23016 raft_consensus.cc:374] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:48.184211 23016 raft_consensus.cc:729] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9109d6505841d49c703c84bf56ca11, State: Initialized, Role: FOLLOWER
I20260810 18:51:48.184744 23016 consensus_queue.cc:260] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.185737 22542 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:48.186420 23016 ts_tablet_manager.cc:1428] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260810 18:51:48.187609 23019 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: No bootstrap required, opened a new log
I20260810 18:51:48.187855 23019 ts_tablet_manager.cc:1397] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260810 18:51:48.190526 23020 raft_consensus.cc:348] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.190896 23020 raft_consensus.cc:374] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:48.190984 23020 raft_consensus.cc:729] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03baadd652a4442cbdad36504e83c524, State: Initialized, Role: FOLLOWER
I20260810 18:51:48.191526 23020 consensus_queue.cc:260] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.192452 22993 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:48.193050 23020 ts_tablet_manager.cc:1428] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 18:51:48.196026 23019 raft_consensus.cc:348] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.196379 23019 raft_consensus.cc:374] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:51:48.196482 23019 raft_consensus.cc:729] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d7114d0aa646afb5c4b6238176e006, State: Initialized, Role: FOLLOWER
I20260810 18:51:48.196987 23019 consensus_queue.cc:260] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.197975 22770 heartbeater.cc:503] Master 127.29.174.254:35015 was elected leader, sending a full tablet report...
I20260810 18:51:48.198405 23019 ts_tablet_manager.cc:1428] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
W20260810 18:51:48.200397 22544 tablet.cc:2369] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:48.306547 23035 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:48.306851 23035 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.308537 23035 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419)
I20260810 18:51:48.315187 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.308987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:48543 (local address 127.29.174.195:34541)
0810 18:51:48.309217 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.309224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.310006 (+ 782us) server_negotiation.cc:407] Connection header received
0810 18:51:48.310152 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.310156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.310216 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.310305 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:48.310934 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.311644 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.313486 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.314251 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.314415 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.314492 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.314808 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.314930 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:51:48.315198 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.309205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:48383 (local address 127.29.174.193:35419)
0810 18:51:48.309413 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.309419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.309912 (+ 493us) server_negotiation.cc:407] Connection header received
0810 18:51:48.310095 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.310100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.310172 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.310238 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:51:48.310851 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.311704 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.313513 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.314270 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.314426 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.314496 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.314832 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.314955 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:51:48.315327 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.308815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.29.174.194:48383)
0810 18:51:48.309796 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.309815 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.309951 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.310278 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.310282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.310307 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:48.310706 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.310719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.311835 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.311839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.313370 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.313382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.314523 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.314530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.314683 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.314705 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.314793 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.314912 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"mutex_wait_us":68,"thread_start_us":607,"threads_started":1}
I20260810 18:51:48.315469 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.308886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34541 (local address 127.29.174.194:48543)
0810 18:51:48.309880 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.309897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:48.310022 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:48.310358 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:48.310364 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:48.310391 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:51:48.310767 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.310776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.311772 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.311777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.313336 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:48.313348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.314490 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.314496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:48.314648 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.314682 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:51:48.314759 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:51:48.314872 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":40,"thread_start_us":688,"threads_started":1}
W20260810 18:51:48.316356 22994 tablet.cc:2369] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:48.316373 22472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95be66a0ec3c4d2aa32cc80b0bde0512" candidate_uuid: "59d7114d0aa646afb5c4b6238176e006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9109d6505841d49c703c84bf56ca11" is_pre_election: true
I20260810 18:51:48.316435 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95be66a0ec3c4d2aa32cc80b0bde0512" candidate_uuid: "59d7114d0aa646afb5c4b6238176e006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03baadd652a4442cbdad36504e83c524" is_pre_election: true
I20260810 18:51:48.316967 22913 raft_consensus.cc:2454] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59d7114d0aa646afb5c4b6238176e006 in term 0.
I20260810 18:51:48.316973 22472 raft_consensus.cc:2454] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59d7114d0aa646afb5c4b6238176e006 in term 0.
I20260810 18:51:48.317612 22573 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11, 59d7114d0aa646afb5c4b6238176e006; no voters:
I20260810 18:51:48.318001 23035 raft_consensus.cc:2781] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:51:48.318122 23035 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:51:48.318228 23035 raft_consensus.cc:3037] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:48.320362 23035 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.320902 23035 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [CANDIDATE]: Term 1 election: Requested vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419)
I20260810 18:51:48.321348 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95be66a0ec3c4d2aa32cc80b0bde0512" candidate_uuid: "59d7114d0aa646afb5c4b6238176e006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03baadd652a4442cbdad36504e83c524"
I20260810 18:51:48.321364 22472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95be66a0ec3c4d2aa32cc80b0bde0512" candidate_uuid: "59d7114d0aa646afb5c4b6238176e006" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9109d6505841d49c703c84bf56ca11"
I20260810 18:51:48.321523 22472 raft_consensus.cc:3037] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:48.321516 22913 raft_consensus.cc:3037] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:51:48.323614 22472 raft_consensus.cc:2454] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59d7114d0aa646afb5c4b6238176e006 in term 1.
I20260810 18:51:48.323657 22913 raft_consensus.cc:2454] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59d7114d0aa646afb5c4b6238176e006 in term 1.
I20260810 18:51:48.323964 22573 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11, 59d7114d0aa646afb5c4b6238176e006; no voters:
I20260810 18:51:48.324216 23035 raft_consensus.cc:2781] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:51:48.324601 23035 raft_consensus.cc:686] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 1 LEADER]: Becoming Leader. State: Replica: 59d7114d0aa646afb5c4b6238176e006, State: Running, Role: LEADER
I20260810 18:51:48.325006 23035 consensus_queue.cc:237] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:48.329392 22211 catalog_manager.cc:5495] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "59d7114d0aa646afb5c4b6238176e006" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } health_report { overall_health: UNKNOWN } } }
W20260810 18:51:48.381599 22771 tablet.cc:2369] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:51:48.385533 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.380203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.29.174.194:41107)
0810 18:51:48.380405 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.380411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.380494 (+ 83us) server_negotiation.cc:407] Connection header received
0810 18:51:48.380711 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.380715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.380787 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.380908 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:51:48.381295 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.382046 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.383701 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.384106 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.384218 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.384619 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:51:48.385070 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:51:48.385194 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.385230 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.385294 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:51:48.392788 22472 raft_consensus.cc:1261] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Refusing update from remote peer 59d7114d0aa646afb5c4b6238176e006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:48.392794 22913 raft_consensus.cc:1261] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Refusing update from remote peer 59d7114d0aa646afb5c4b6238176e006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:51:48.393622 23035 consensus_queue.cc:1035] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:48.393872 23039 consensus_queue.cc:1035] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:51:48.405627 23045 mvcc.cc:205] Tried to move back new op lower bound from 7317044872768364544 to 7317044872502902784. Current Snapshot: MvccSnapshot[applied={T|T < 7317044872768364544}]
I20260810 18:51:48.416684 22212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:48.416888 22212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:51:48.420150 22212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Sending DeleteTablet for 3 replicas of tablet 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:48.421221 22660 tablet_service.cc:1514] Processing DeleteTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:48 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:44574
I20260810 18:51:48.421216 22879 tablet_service.cc:1514] Processing DeleteTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:48 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:34954
I20260810 18:51:48.421394 22430 tablet_service.cc:1514] Processing DeleteTablet for tablet 95be66a0ec3c4d2aa32cc80b0bde0512 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:51:48 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:34238
I20260810 18:51:48.422286 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 22290
I20260810 18:51:48.441691 23015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.423063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.29.174.193:35419)
0810 18:51:48.423230 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.423236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.423325 (+ 89us) server_negotiation.cc:407] Connection header received
0810 18:51:48.423486 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.423491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.423566 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.423651 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:48.425402 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.426646 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.428549 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.429270 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.429389 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.432853 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:48.432870 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:48.432886 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:48.432928 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:48.436877 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.437731 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.437742 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.437746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.437917 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.438504 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.438510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.438514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.440319 (+ 1805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:48.440715 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.441061 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.441317 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:51:48.442909 22529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:48.444958 23060 tablet_replica.cc:330] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: stopping tablet replica
I20260810 18:51:48.445513 23061 tablet_replica.cc:330] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: stopping tablet replica
I20260810 18:51:48.446866 23061 raft_consensus.cc:2229] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:51:48.446746 23060 raft_consensus.cc:2229] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:48.448042 23061 raft_consensus.cc:2258] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:48.448588 23060 raft_consensus.cc:2258] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:48.450314 23061 ts_tablet_manager.cc:1905] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:48.451004 23060 ts_tablet_manager.cc:1905] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:51:48.457374 23060 ts_tablet_manager.cc:1918] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:48.457536 23061 ts_tablet_manager.cc:1918] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:51:48.457602 23060 log.cc:1196] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:48.458181 23061 log.cc:1196] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:48.458180 23060 ts_tablet_manager.cc:1939] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 03baadd652a4442cbdad36504e83c524: Deleting consensus metadata
I20260810 18:51:48.458905 23061 ts_tablet_manager.cc:1939] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 59d7114d0aa646afb5c4b6238176e006: Deleting consensus metadata
I20260810 18:51:48.460783 22184 catalog_manager.cc:4841] TS 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): tablet 95be66a0ec3c4d2aa32cc80b0bde0512 (table test-table [id=d3716771fea8422fb0201380588a8010]) successfully deleted
I20260810 18:51:48.461238 22189 catalog_manager.cc:4841] TS 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): tablet 95be66a0ec3c4d2aa32cc80b0bde0512 (table test-table [id=d3716771fea8422fb0201380588a8010]) successfully deleted
I20260810 18:51:48.571313 23064 tablet_replica.cc:330] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: stopping tablet replica
I20260810 18:51:48.571724 23064 raft_consensus.cc:2229] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:51:48.572139 23064 raft_consensus.cc:2258] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:51:48.573347 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 22290
W20260810 18:51:48.597795 22185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:35419 (error 108)
I20260810 18:51:48.597957 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 22548
I20260810 18:51:48.599874 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.599366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34252)
0810 18:51:48.599585 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.599695 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260810 18:51:48.600121 22185 catalog_manager.cc:4584] TS 4b9109d6505841d49c703c84bf56ca11 (127.29.174.193:35419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95be66a0ec3c4d2aa32cc80b0bde0512: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
I20260810 18:51:48.614012 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.598950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.29.174.194:41107)
0810 18:51:48.599097 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.599101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.599118 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:51:48.599180 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.599183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.599247 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.599342 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:48.600972 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.601794 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.603158 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.603767 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.603887 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.607243 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:48.607258 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:48.607275 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:48.607318 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:48.611376 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.612055 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.612063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.612065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.612205 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.612606 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.612609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.612612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.613019 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:48.613292 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.613600 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.613761 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:51:48.614712 22760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:48.653549 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.653050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34254)
0810 18:51:48.653264 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.653382 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:51:48.654627 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.654199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34262)
0810 18:51:48.654381 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.654492 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 18:51:48.715161 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.714608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34278)
0810 18:51:48.714836 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.714940 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:51:48.716157 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.715816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34286)
0810 18:51:48.715951 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.716028 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 18:51:48.734478 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 22548
I20260810 18:51:48.756717 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 22778
I20260810 18:51:48.769672 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.757413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34960 (local address 127.29.174.195:34541)
0810 18:51:48.757615 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:48.757621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:48.757670 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:51:48.757869 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:48.757875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:48.757950 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:48.758035 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:48.759452 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.760164 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.761445 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:48.761837 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:48.761947 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:48.764570 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:48.764584 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:48.764599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:48.764629 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:48.767011 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.767680 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.767687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.767690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.767818 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:48.768172 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:48.768177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:48.768180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:48.768539 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:48.768827 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:48.769141 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 18:51:48.769356 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:51:48.770262 22985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:48.796430 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.795996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34296)
0810 18:51:48.796185 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.796271 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:51:48.797477 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.797095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34304)
0810 18:51:48.797282 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.797372 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 18:51:48.856213 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 22778
I20260810 18:51:48.879346 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 22159
I20260810 18:51:48.880452 22280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:51:48.992875 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:48.992267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35419 (local address 127.0.0.1:34316)
0810 18:51:48.992482 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:48.992615 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:35419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:51:48.993019 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 22159
I20260810 18:51:49.017689 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--webserver_interface=127.29.174.254
--webserver_port=38945
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:49.339834 23080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:49.340090 23080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:49.340196 23080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:49.347355 23080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:51:49.347452 23080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:49.347505 23080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:49.347561 23080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:51:49.347606 23080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:51:49.357095 23080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=38945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:49.358211 23080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:49.359441 23080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:49.370502 23086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:49.370545 23092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:49.370566 23090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:49.371703 23080 server_base.cc:1034] running on GCE node
I20260810 18:51:49.372403 23080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:49.373667 23080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:49.374866 23080 hybrid_clock.cc:648] HybridClock initialized: now 1786387909374821 us; error 28 us; skew 500 ppm
I20260810 18:51:49.375198 23080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:49 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:49.378206 23080 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:51:49.379597 23080 webserver.cc:460] Webserver started at http://127.29.174.254:38945/ using document root <none> and password file <none>
I20260810 18:51:49.380244 23080 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:49.380342 23080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:49.385385 23080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:51:49.388037 23107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:49.389236 23080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:51:49.389387 23080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6e0eec2881c3421db2a74b15b9f0b59c"
format_stamp: "Formatted at 2026-08-10 18:51:43 on dist-test-slave-6zbq"
server_key: "3804717ca5a12646d1dff34b93021642"
server_key_iv: "db0724858649849a0961b61fd31f72b0"
server_key_version: "encryptionkey@0"
I20260810 18:51:49.390381 23080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:49.405020 23080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:49.408939 23080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:49.409201 23080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:49.426082 23080 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35015
I20260810 18:51:49.426101 23189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35015 every 8 connection(s)
I20260810 18:51:49.427436 23080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:51:49.430912 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 23080
I20260810 18:51:49.431895 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:35419
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=33657
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:51:49.434899 23190 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:51:49.438949 23190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap starting.
I20260810 18:51:49.460414 23190 log.cc:824] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:49.476249 23190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:49.476754 23190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c: Bootstrap complete.
I20260810 18:51:49.485556 23190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:49.486433 23190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Initialized, Role: FOLLOWER
I20260810 18:51:49.486992 23190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:49.487201 23190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:51:49.487317 23190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:51:49.487468 23190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:51:49.491112 23190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:49.491568 23190 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0eec2881c3421db2a74b15b9f0b59c; no voters:
I20260810 18:51:49.492187 23190 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:51:49.492324 23200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:51:49.493399 23200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [term 3 LEADER]: Becoming Leader. State: Replica: 6e0eec2881c3421db2a74b15b9f0b59c, State: Running, Role: LEADER
I20260810 18:51:49.493981 23200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } }
I20260810 18:51:49.494364 23190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:51:49.496570 23202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:49.496912 23202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:49.498378 23206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0eec2881c3421db2a74b15b9f0b59c. Latest consensus state: current_term: 3 leader_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0eec2881c3421db2a74b15b9f0b59c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35015 } } }
I20260810 18:51:49.498658 23206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0eec2881c3421db2a74b15b9f0b59c [sys.catalog]: This master's current role is: LEADER
I20260810 18:51:49.498687 23213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:51:49.507331 23213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:51:49.508872 23213 catalog_manager.cc:1259] Loaded cluster ID: 6556aa02713f46e8a0435e16caebfbaa
I20260810 18:51:49.509023 23213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 18:51:49.511283 23213 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6ea24f89ad google::LogMessage::Fail() at ??:0
@ 0x7f6ea24fb88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6ea24f84de google::LogMessage::Flush() at ??:0
@ 0x7f6ea24f9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6eb7f2b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6eb7f28976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6eb7fb0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6eb7fb0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6ea5144345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6ea514791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6ea5147799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6ea5122eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6ea83fc6db start_thread at ??:0
@ 0x7f6ea139261f clone at ??:0
W20260810 18:51:49.795706 23195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:49.795990 23195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:49.796124 23195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:49.803414 23195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:49.803527 23195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:49.803684 23195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:51:49.813436 23195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:35419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=33657
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:49.814772 23195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:49.816058 23195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:49.828249 23274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:49.828245 23276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:49.828253 23279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:49.829095 23195 server_base.cc:1034] running on GCE node
I20260810 18:51:49.829663 23195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:49.830530 23195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:49.831722 23195 hybrid_clock.cc:648] HybridClock initialized: now 1786387909831681 us; error 37 us; skew 500 ppm
I20260810 18:51:49.832001 23195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:49 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:49.839607 23195 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:51:49.841154 23195 webserver.cc:460] Webserver started at http://127.29.174.193:33657/ using document root <none> and password file <none>
I20260810 18:51:49.841823 23195 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:49.841943 23195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:49.847231 23195 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:51:49.850131 23294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:49.851471 23195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:51:49.851629 23195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "4b9109d6505841d49c703c84bf56ca11"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "b2e3cc2cee2db6b67d3b56eec081011b"
server_key_iv: "9b0bf7614372df3da897c91203104e2a"
server_key_version: "encryptionkey@0"
I20260810 18:51:49.852567 23195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:49.869786 23195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:49.874168 23195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:49.874475 23195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:49.875432 23195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:49.877147 23307 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 18:51:49.882726 23195 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:51:49.882839 23195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 18:51:49.882958 23195 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:51:49.886062 23195 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:51:49.886142 23195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260810 18:51:49.886608 23307 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Bootstrap starting.
I20260810 18:51:49.895144 23307 log.cc:824] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Log is configured to *not* fsync() on all Append() calls
I20260810 18:51:49.909119 23307 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:51:49.909976 23307 tablet_bootstrap.cc:492] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Bootstrap complete.
I20260810 18:51:49.910537 23307 ts_tablet_manager.cc:1397] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.010s
I20260810 18:51:49.920081 23307 raft_consensus.cc:348] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:49.921514 23307 raft_consensus.cc:729] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9109d6505841d49c703c84bf56ca11, State: Initialized, Role: FOLLOWER
I20260810 18:51:49.922142 23307 consensus_queue.cc:260] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:49.923317 23195 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35419
I20260810 18:51:49.923322 23485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35419 every 8 connection(s)
I20260810 18:51:49.924458 23195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:51:49.926496 23307 ts_tablet_manager.cc:1428] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260810 18:51:49.928839 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:49.927072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:55821)
0810 18:51:49.927830 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:49.928042 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":310,"threads_started":1}
W20260810 18:51:49.929512 23486 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:35015 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:35015: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
I20260810 18:51:49.930249 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:49.929950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:44749)
0810 18:51:49.930072 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:49.930140 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 18:51:49.931118 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:49.930790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:55567)
0810 18:51:49.930920 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:49.930988 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:51:49.931363 23486 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:51:49.933143 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 23195
I20260810 18:51:49.933770 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:41107
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=36753
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:50.258654 23496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:50.258908 23496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:50.259037 23496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:50.266083 23496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:50.266429 23496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:50.266620 23496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:51:50.276266 23496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:41107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=36753
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:50.277416 23496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:50.278749 23496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:50.291069 23504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:50.291128 23505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:50.291122 23509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:50.292119 23496 server_base.cc:1034] running on GCE node
I20260810 18:51:50.292644 23496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:50.293466 23496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:50.294701 23496 hybrid_clock.cc:648] HybridClock initialized: now 1786387910294646 us; error 40 us; skew 500 ppm
I20260810 18:51:50.295019 23496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:50 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:50.298103 23496 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:51:50.299532 23496 webserver.cc:460] Webserver started at http://127.29.174.194:36753/ using document root <none> and password file <none>
I20260810 18:51:50.300148 23496 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:50.300266 23496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:50.305510 23496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 18:51:50.308203 23519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.309475 23496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 18:51:50.309629 23496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "59d7114d0aa646afb5c4b6238176e006"
format_stamp: "Formatted at 2026-08-10 18:51:44 on dist-test-slave-6zbq"
server_key: "c11c5d8d2d1a25732743799eaa45108a"
server_key_iv: "2a997a4c7693809124d6ab5bde7b42ff"
server_key_version: "encryptionkey@0"
I20260810 18:51:50.310626 23496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:50.351346 23496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:50.355512 23496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:50.355769 23496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:50.356778 23496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:50.358165 23496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:50.358264 23496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.358387 23496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:50.358460 23496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.390715 23496 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:41107
I20260810 18:51:50.390784 23729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:41107 every 8 connection(s)
I20260810 18:51:50.392128 23496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:51:50.396579 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.394884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:46183)
0810 18:51:50.395583 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.395811 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":259,"threads_started":1}
W20260810 18:51:50.397233 23731 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:35015 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:35015: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
I20260810 18:51:50.398078 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.397694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:60599)
0810 18:51:50.397871 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.397950 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:51:50.398945 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.398643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:39095)
0810 18:51:50.398762 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.398841 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 18:51:50.399166 23731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:51:50.400386 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 23496
I20260810 18:51:50.400979 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:34541
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:35015
--webserver_port=33597
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:51:50.726958 23735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:51:50.727200 23735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:51:50.727335 23735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:51:50.734174 23735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:51:50.734273 23735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:51:50.734433 23735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:51:50.743605 23735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:34541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=33597
--tserver_master_addrs=127.29.174.254:35015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:51:50.744746 23735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:51:50.746006 23735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:51:50.758916 23745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:50.758888 23750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:51:50.758852 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:51:50.759531 23735 server_base.cc:1034] running on GCE node
I20260810 18:51:50.760113 23735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:51:50.760965 23735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:51:50.762163 23735 hybrid_clock.cc:648] HybridClock initialized: now 1786387910762114 us; error 36 us; skew 500 ppm
I20260810 18:51:50.762481 23735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:51:50 dist-test-slave-6zbq krb5kdc[21247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387910, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:51:50.765317 23735 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:51:50.766561 23735 webserver.cc:460] Webserver started at http://127.29.174.195:33597/ using document root <none> and password file <none>
I20260810 18:51:50.767174 23735 fs_manager.cc:359] Metadata directory not provided
I20260810 18:51:50.767282 23735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:51:50.772066 23735 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 18:51:50.774545 23766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.775713 23735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:51:50.775856 23735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "03baadd652a4442cbdad36504e83c524"
format_stamp: "Formatted at 2026-08-10 18:51:45 on dist-test-slave-6zbq"
server_key: "529a034db65be866eca9b3564ca01dad"
server_key_iv: "7c7153034f685022320047241d5f77c7"
server_key_version: "encryptionkey@0"
I20260810 18:51:50.776779 23735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:51:50.805585 23735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:51:50.809262 23735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:51:50.809501 23735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:51:50.810415 23735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:51:50.811678 23735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:51:50.811764 23735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.811864 23735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:51:50.811937 23735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:51:50.841395 23735 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:34541
I20260810 18:51:50.841463 23964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:34541 every 8 connection(s)
I20260810 18:51:50.842854 23735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:51:50.847657 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.845822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:51187)
0810 18:51:50.846610 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.846836 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":304,"threads_started":1}
I20260810 18:51:50.848241 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 23735
W20260810 18:51:50.848343 23965 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:35015 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:35015: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
I20260810 18:51:50.849117 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.848800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:48065)
0810 18:51:50.848945 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.849014 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:51:50.849912 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.849597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:60391)
0810 18:51:50.849731 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.849808 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:51:50.850147 23965 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:51:50.932498 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:50.931803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:53097)
0810 18:51:50.932206 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:50.932300 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":185,"threads_started":1}
I20260810 18:51:51.400390 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.399614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:36197)
0810 18:51:51.400048 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:51.400172 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":209,"threads_started":1}
I20260810 18:51:51.550437 23974 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:51.550776 23974 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:51.552311 23974 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
Aug 10 18:51:51 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
Aug 10 18:51:51 dist-test-slave-6zbq krb5kdc[21247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387909, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:51:51.584776 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.552880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:50795 (local address 127.29.174.195:34541)
0810 18:51:51.564642 (+ 11762us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:51.564661 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:51.565654 (+ 993us) server_negotiation.cc:407] Connection header received
0810 18:51:51.567414 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:51.567426 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:51.567794 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:51.568054 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:51.569913 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.571354 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573023 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.573643 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573808 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:51.577546 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:51.577557 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:51.577574 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:51.577616 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:51.581158 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582055 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:51.582064 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:51.582071 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:51.582228 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582671 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:51.582677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:51.582680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:51.583131 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:51.583293 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:51.584058 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 18:51:51.584362 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11543,"thread_start_us":11416,"threads_started":1}
I20260810 18:51:51.584781 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.552920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:37011 (local address 127.29.174.194:41107)
0810 18:51:51.565727 (+ 12807us) server_negotiation.cc:206] Beginning negotiation
0810 18:51:51.565740 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:51:51.565786 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:51:51.567518 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:51:51.567527 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:51:51.567842 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:51:51.568111 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:51:51.569909 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.571274 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573023 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.573642 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573807 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:51.577873 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:51:51.577880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:51:51.577890 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:51:51.577923 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:51:51.581149 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582057 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:51.582064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:51.582069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:51.582217 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582679 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:51:51.582683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:51:51.582687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:51:51.583141 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:51:51.583294 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:51:51.584045 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 18:51:51.584415 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12627,"thread_start_us":12480,"threads_started":1}
I20260810 18:51:51.584786 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.552625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:41107 (local address 127.29.174.193:37011)
0810 18:51:51.565019 (+ 12394us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:51.565082 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:51.565740 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:51.568232 (+ 2492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:51.568242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:51.569066 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:51.569729 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:51.569752 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.571447 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.571453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:51.572890 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:51.572903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.573841 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:51.574110 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:51.574898 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:51.574922 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:51.577637 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:51.581292 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:51.581300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:51.581313 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:51.581938 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:51.582357 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:51.582366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:51.582568 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:51.583276 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:51.583284 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:51.583567 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 18:51:51.583796 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 18:51:51.584311 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12230,"mutex_wait_us":278,"spinlock_wait_cycles":13056,"thread_start_us":12481,"threads_started":1}
I20260810 18:51:51.584824 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.552676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:34541 (local address 127.29.174.193:50795)
0810 18:51:51.565449 (+ 12773us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:51.565467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:51:51.565731 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:51:51.568232 (+ 2501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:51:51.568242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:51:51.569069 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:51:51.569728 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:51.569752 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.571511 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.571515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:51.572888 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:51:51.572901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:51:51.573890 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:51:51.573899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:51:51.574112 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:51:51.574897 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:51:51.574924 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:51:51.577254 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:51:51.581316 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:51.581319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:51.581321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:51.581930 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:51.582368 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:51:51.582374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:51:51.582378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:51:51.582560 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:51:51.583277 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:51:51.583284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:51:51.583567 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 18:51:51.583797 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 18:51:51.584454 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12655,"mutex_wait_us":271,"spinlock_wait_cycles":45312,"thread_start_us":12008,"threads_started":1}
W20260810 18:51:51.587984 23301 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
W20260810 18:51:51.588429 23295 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:51.588559 23295 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11; no voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006
I20260810 18:51:51.588929 23974 raft_consensus.cc:2726] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:51.851581 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.850686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:50279)
0810 18:51:51.851210 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:51.851357 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":243,"threads_started":1}
I20260810 18:51:51.933683 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:51.933298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:53949)
0810 18:51:51.933466 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:51.933558 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:51:52.401840 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:52.401171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:59519)
0810 18:51:52.401619 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:52.401697 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":219,"threads_started":1}
I20260810 18:51:52.852962 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:52.852347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:54665)
0810 18:51:52.852744 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:52.852820 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":204,"threads_started":1}
I20260810 18:51:52.935155 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:52.934496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:55339)
0810 18:51:52.934915 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:52.934991 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":208,"threads_started":1}
I20260810 18:51:53.403193 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:53.402581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:56467)
0810 18:51:53.402963 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:53.403037 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":199,"threads_started":1}
I20260810 18:51:53.758435 23985 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:53.758610 23985 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:53.759264 23985 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
W20260810 18:51:53.760177 23295 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
W20260810 18:51:53.760339 23301 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:53.760483 23301 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11; no voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006
I20260810 18:51:53.760717 23985 raft_consensus.cc:2726] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:53.854363 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:53.853708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:36869)
0810 18:51:53.854105 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:53.854179 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":199,"threads_started":1}
I20260810 18:51:53.936656 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:53.935895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:51407)
0810 18:51:53.936364 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:53.936475 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":220,"threads_started":1}
I20260810 18:51:54.404569 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:54.403917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:48289)
0810 18:51:54.404330 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:54.404414 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":192,"threads_started":1}
I20260810 18:51:54.855821 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:54.855100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:54793)
0810 18:51:54.855527 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:54.855635 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":204,"threads_started":1}
I20260810 18:51:54.938217 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:54.937555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:51629)
0810 18:51:54.937977 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:54.938056 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":204,"threads_started":1}
I20260810 18:51:55.406014 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:55.405352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:33837)
0810 18:51:55.405799 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:55.405875 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260810 18:51:55.857342 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:55.856640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:42569)
0810 18:51:55.857092 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:55.857179 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":226,"threads_started":1}
I20260810 18:51:55.939723 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:55.939039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:37893)
0810 18:51:55.939484 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:55.939562 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":251,"threads_started":1}
I20260810 18:51:56.239415 24004 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:51:56.239615 24004 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:51:56.240254 24004 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
W20260810 18:51:56.241140 23295 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
W20260810 18:51:56.241302 23301 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:51:56.241441 23301 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11; no voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006
I20260810 18:51:56.241699 24004 raft_consensus.cc:2726] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:51:56.407444 24005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:56.406761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:38083)
0810 18:51:56.407193 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:56.407275 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":221,"threads_started":1}
I20260810 18:51:56.858843 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:56.858116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:60313)
0810 18:51:56.858591 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:56.858677 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260810 18:51:56.941251 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:56.940503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:41009)
0810 18:51:56.940955 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:56.941040 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":252,"threads_started":1}
I20260810 18:51:57.409039 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:57.408265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:60595)
0810 18:51:57.408762 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:57.408851 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":216,"threads_started":1}
I20260810 18:51:57.860489 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:57.859692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:60427)
0810 18:51:57.860230 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:57.860311 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":234,"threads_started":1}
I20260810 18:51:57.942912 24014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:57.942080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:55827)
0810 18:51:57.942568 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:57.942672 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":277,"threads_started":1}
I20260810 18:51:58.410457 24015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:58.409797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:58601)
0810 18:51:58.410173 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:58.410269 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":187,"threads_started":1}
I20260810 18:51:58.861989 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:58.861327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:56833)
0810 18:51:58.861724 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:58.861804 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":202,"threads_started":1}
I20260810 18:51:58.944406 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:58.943736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:59127)
0810 18:51:58.944159 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:58.944240 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":203,"threads_started":1}
I20260810 18:51:59.411938 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:59.411226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:53167)
0810 18:51:59.411707 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:59.411793 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":216,"threads_started":1}
I20260810 18:51:59.863441 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:59.862762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:35353)
0810 18:51:59.863216 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:59.863292 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":213,"threads_started":1}
I20260810 18:51:59.945955 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:51:59.945277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:35467)
0810 18:51:59.945717 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 18:51:59.945803 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":235,"threads_started":1}
I20260810 18:52:00.413516 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:00.412785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:45551)
0810 18:52:00.413223 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:00.413304 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":225,"threads_started":1}
I20260810 18:52:00.592217 24025 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:00.592423 24025 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:52:00.593045 24025 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
W20260810 18:52:00.593935 23295 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
W20260810 18:52:00.594096 23301 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:52:00.594209 23301 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11; no voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006
I20260810 18:52:00.594465 24025 raft_consensus.cc:2726] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:00.864962 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:00.864255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:44259)
0810 18:52:00.864687 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:00.864767 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":237,"threads_started":1}
I20260810 18:52:00.947507 24033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:00.946753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:40811)
0810 18:52:00.947214 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:00.947331 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":212,"threads_started":1}
I20260810 18:52:01.415059 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.414300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:60259)
0810 18:52:01.414769 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.414882 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":258,"threads_started":1}
I20260810 18:52:01.866459 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.865733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:37907)
0810 18:52:01.866171 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.866263 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":229,"threads_started":1}
I20260810 18:52:01.948911 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:01.948259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:58015)
0810 18:52:01.948682 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:01.948759 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":229,"threads_started":1}
I20260810 18:52:02.416494 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.415822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:52081)
0810 18:52:02.416243 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.416323 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":204,"threads_started":1}
I20260810 18:52:02.867870 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.867223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:45061)
0810 18:52:02.867628 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.867702 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":214,"threads_started":1}
I20260810 18:52:02.950430 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:02.949701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:44727)
0810 18:52:02.950113 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:02.950206 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":214,"threads_started":1}
I20260810 18:52:03.417899 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.417232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:36321)
0810 18:52:03.417646 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.417743 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260810 18:52:03.869270 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.868600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:55921)
0810 18:52:03.869001 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.869085 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":207,"threads_started":1}
I20260810 18:52:03.872373 24048 raft_consensus.cc:482] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:03.872540 24048 raft_consensus.cc:504] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03baadd652a4442cbdad36504e83c524" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 34541 } } peers { permanent_uuid: "59d7114d0aa646afb5c4b6238176e006" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 41107 } } peers { permanent_uuid: "4b9109d6505841d49c703c84bf56ca11" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35419 } }
I20260810 18:52:03.873186 24048 leader_election.cc:288] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541), 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107)
W20260810 18:52:03.873930 23295 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03baadd652a4442cbdad36504e83c524 (127.29.174.195:34541): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
W20260810 18:52:03.874066 23301 leader_election.cc:341] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59d7114d0aa646afb5c4b6238176e006 (127.29.174.194:41107): Not found: Tablet not found: 95be66a0ec3c4d2aa32cc80b0bde0512
I20260810 18:52:03.874238 23301 leader_election.cc:302] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b9109d6505841d49c703c84bf56ca11; no voters: 03baadd652a4442cbdad36504e83c524, 59d7114d0aa646afb5c4b6238176e006
I20260810 18:52:03.874512 24048 raft_consensus.cc:2726] T 95be66a0ec3c4d2aa32cc80b0bde0512 P 4b9109d6505841d49c703c84bf56ca11 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:52:03.951787 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:03.951149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:58501)
0810 18:52:03.951516 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:03.951607 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":186,"threads_started":1}
I20260810 18:52:04.419318 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.418651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:58905)
0810 18:52:04.419047 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.419122 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":208,"threads_started":1}
I20260810 18:52:04.870759 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.870034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:47389)
0810 18:52:04.870461 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.870552 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":213,"threads_started":1}
I20260810 18:52:04.953439 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:04.952787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.193:58681)
0810 18:52:04.953186 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:04.953263 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":202,"threads_started":1}
I20260810 18:52:05.420966 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.420254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.194:37713)
0810 18:52:05.420681 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:05.420798 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":201,"threads_started":1}
I20260810 18:52:05.850558 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 23195
I20260810 18:52:05.865037 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.851505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.29.174.193:35419)
0810 18:52:05.852008 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:05.852018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:05.852051 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:52:05.852114 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:05.852120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:05.852396 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:05.852563 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:05.854801 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:05.855766 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:05.856931 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:05.857610 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:05.857711 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:05.860164 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:05.860176 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:05.860188 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:05.860232 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:05.862724 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:05.863327 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:05.863333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:05.863335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:05.863478 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:05.863750 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:05.863753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:05.863755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:05.864085 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:05.864314 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:05.864524 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:52:05.864710 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":225,"threads_started":1}
I20260810 18:52:05.866185 23476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:05.872498 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.871794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35015 (local address 127.29.174.195:37849)
0810 18:52:05.872248 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:05.872343 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":234,"threads_started":1}
I20260810 18:52:05.952469 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 23195
I20260810 18:52:05.975714 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 23496
I20260810 18:52:05.988181 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:05.976442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.29.174.194:41107)
0810 18:52:05.976908 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:05.976912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:05.976929 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:52:05.976985 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:05.976988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:05.977047 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:05.977134 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:05.978381 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:05.979151 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:05.980386 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:05.980835 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:05.980930 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:05.983540 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:05.983546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:05.983549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:05.983579 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:05.986059 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:05.986679 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:05.986682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:05.986684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:05.986812 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:05.987080 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:05.987083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:05.987085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:05.987324 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:05.987521 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:05.987685 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 18:52:05.987848 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260810 18:52:05.988848 23722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:06.070147 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 23496
I20260810 18:52:06.091190 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 23735
I20260810 18:52:06.105118 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:06.091868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.29.174.195:34541)
0810 18:52:06.092404 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:06.092409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:06.092424 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:52:06.092481 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:06.092485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:06.092545 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:06.092644 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:06.094106 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:06.094954 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:06.096165 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:06.096597 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:06.096709 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:06.099430 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:06.099435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:06.099439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:06.099470 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:06.101879 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:06.102695 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:06.102701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:06.102705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:06.102900 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:06.103310 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:06.103316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:06.103319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:06.103664 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:06.103940 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:06.104474 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 18:52:06.104704 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":263,"threads_started":1}
I20260810 18:52:06.105958 23958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:06.186684 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 23735
2026-08-10T18:52:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24582 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:52:06 dist-test-slave-6zbq krb5kdc[24076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:52:06 dist-test-slave-6zbq krb5kdc[24076](info): set up 2 sockets
Aug 10 18:52:06 dist-test-slave-6zbq krb5kdc[24076](info): commencing operation
krb5kdc: starting...
W20260810 18:52:07.419299 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:52:07 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:52:07.649713 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:07.962004 24094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:07.962263 24094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:07.962407 24094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:07.969237 24094 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:52:07.969326 24094 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:52:07.969388 24094 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:52:07.969462 24094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:07.969532 24094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:07.969619 24094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:07.969678 24094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:07.969750 24094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:07.978911 24094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:07.979916 24094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:07.981043 24094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:07.991528 24103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:07.991493 24105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:07.991461 24111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:07.992642 24094 server_base.cc:1034] running on GCE node
I20260810 18:52:07.993284 24094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:07.994418 24094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:07.995622 24094 hybrid_clock.cc:648] HybridClock initialized: now 1786387927995567 us; error 31 us; skew 500 ppm
I20260810 18:52:07.995913 24094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:07 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:07.998790 24094 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:52:08.000031 24094 webserver.cc:460] Webserver started at http://127.29.174.254:45747/ using document root <none> and password file <none>
I20260810 18:52:08.000628 24094 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:08.000725 24094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:08.000993 24094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:08.003701 24094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "462e0e7ccb8248f8a51cd7589cdf4554"
format_stamp: "Formatted at 2026-08-10 18:52:07 on dist-test-slave-6zbq"
server_key: "d937507dd69fe42d6cd3de10767a9793"
server_key_iv: "6abea1c51e9a6f28c355688275313b8f"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.004415 24094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "462e0e7ccb8248f8a51cd7589cdf4554"
format_stamp: "Formatted at 2026-08-10 18:52:07 on dist-test-slave-6zbq"
server_key: "d937507dd69fe42d6cd3de10767a9793"
server_key_iv: "6abea1c51e9a6f28c355688275313b8f"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.009055 24094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 18:52:08.012122 24123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:08.013234 24094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:52:08.013391 24094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "462e0e7ccb8248f8a51cd7589cdf4554"
format_stamp: "Formatted at 2026-08-10 18:52:07 on dist-test-slave-6zbq"
server_key: "d937507dd69fe42d6cd3de10767a9793"
server_key_iv: "6abea1c51e9a6f28c355688275313b8f"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.013568 24094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:08.030416 24094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:08.034061 24094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:08.034271 24094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:08.050169 24094 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39349
I20260810 18:52:08.050202 24197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39349 every 8 connection(s)
I20260810 18:52:08.051322 24094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:08.054570 24198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:08.058483 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 24094
I20260810 18:52:08.058704 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:52:08.059510 30395 external_mini_cluster.cc:1428] Setting key f31d7a57fcb5ce0746f9f43a5c50bdb9
I20260810 18:52:08.065357 24198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap starting.
I20260810 18:52:08.068607 24198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:08.069710 24198 log.cc:824] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:08.072670 24198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: No bootstrap required, opened a new log
Aug 10 18:52:08 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:08.081496 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.061320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.29.174.254:39349)
0810 18:52:08.062008 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:08.062018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:08.062059 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:52:08.064004 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:08.064025 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:08.064338 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:08.064658 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:08.066420 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.067682 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.069414 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.070067 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.070246 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:08.074959 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:08.074967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:08.074979 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:08.075012 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:08.077882 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:08.078655 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:08.078661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:08.078669 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:08.078835 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:08.079191 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:08.079194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:08.079196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:08.079679 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:08.079909 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:08.080292 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 18:52:08.080555 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":320,"threads_started":1}
I20260810 18:52:08.081315 24198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:08.081663 24198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:08.081804 24198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Initialized, Role: FOLLOWER
I20260810 18:52:08.082430 24198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:08.082707 24198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:08.082854 24198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:08.083009 24198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:08.085211 24198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:08.085629 24198 leader_election.cc:302] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462e0e7ccb8248f8a51cd7589cdf4554; no voters:
I20260810 18:52:08.086112 24198 leader_election.cc:288] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:52:08.086237 24204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:08.087258 24204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 LEADER]: Becoming Leader. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Running, Role: LEADER
I20260810 18:52:08.087783 24204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:08.087954 24198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:08.090370 24209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:08.090366 24210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462e0e7ccb8248f8a51cd7589cdf4554. Latest consensus state: current_term: 1 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:08.090833 24210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:08.090830 24209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:08.091636 24218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:08.096402 24218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:08.105746 24218 catalog_manager.cc:1347] Generated new cluster ID: df7d73688feb45378cc62205b99971d1
I20260810 18:52:08.105840 24218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:08.117535 24218 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:52:08.118732 24218 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:08.129462 24218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Generated new TSK 0
I20260810 18:52:08.130155 24218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:52:08.201422 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:08.516654 24239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:08.516881 24239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:08.517000 24239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:08.524277 24239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:08.524379 24239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:08.524523 24239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:52:08.533849 24239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:08.534911 24239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:08.536064 24239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:08.548008 24259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:08.548100 24255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:08.548048 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:08.548825 24239 server_base.cc:1034] running on GCE node
I20260810 18:52:08.549345 24239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:08.550128 24239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:08.551311 24239 hybrid_clock.cc:648] HybridClock initialized: now 1786387928551239 us; error 66 us; skew 500 ppm
I20260810 18:52:08.551563 24239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:08 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:08.554554 24239 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:52:08.555787 24239 webserver.cc:460] Webserver started at http://127.29.174.193:40319/ using document root <none> and password file <none>
I20260810 18:52:08.556383 24239 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:08.556491 24239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:08.556775 24239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:08.559470 24239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "f2a8e5c75c484867be80f4cb36c02207"
format_stamp: "Formatted at 2026-08-10 18:52:08 on dist-test-slave-6zbq"
server_key: "107fb834a6b20502ab1ee141ff75f10a"
server_key_iv: "a48d2dd65cbf017717ef79b833cb3d0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.560194 24239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "f2a8e5c75c484867be80f4cb36c02207"
format_stamp: "Formatted at 2026-08-10 18:52:08 on dist-test-slave-6zbq"
server_key: "107fb834a6b20502ab1ee141ff75f10a"
server_key_iv: "a48d2dd65cbf017717ef79b833cb3d0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.564851 24239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:52:08.567911 24281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:08.569059 24239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:52:08.569188 24239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "f2a8e5c75c484867be80f4cb36c02207"
format_stamp: "Formatted at 2026-08-10 18:52:08 on dist-test-slave-6zbq"
server_key: "107fb834a6b20502ab1ee141ff75f10a"
server_key_iv: "a48d2dd65cbf017717ef79b833cb3d0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:08.569365 24239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:08.586736 24239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:08.590431 24239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:08.590660 24239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:08.591473 24239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:08.592736 24239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:08.592818 24239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:08.592936 24239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:08.592999 24239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:08.626881 24239 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35481
I20260810 18:52:08.626936 24470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35481 every 8 connection(s)
I20260810 18:52:08.627997 24239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:08.635882 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 24239
I20260810 18:52:08.636111 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:52:08.636931 30395 external_mini_cluster.cc:1428] Setting key 3a55921e8c982f288134cb6bd55fdb20
Aug 10 18:52:08 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387928, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:08.646874 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.630250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:55557 (local address 127.29.174.254:39349)
0810 18:52:08.630693 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:08.630699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:08.631564 (+ 865us) server_negotiation.cc:407] Connection header received
0810 18:52:08.633242 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:08.633246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:08.633313 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:08.633405 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:08.634741 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.635465 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.637618 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.638102 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.638194 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:08.641418 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:08.641424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:08.641428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:08.641461 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:08.644186 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:08.644819 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:08.644824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:08.644827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:08.644963 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:08.645312 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:08.645316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:08.645319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:08.645631 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:08.645774 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:08.646333 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 18:52:08.646558 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":222,"threads_started":1}
I20260810 18:52:08.647487 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:08.630781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:55557)
0810 18:52:08.631432 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:08.631466 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:08.633012 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:08.633579 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:08.633586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:08.634035 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:08.634610 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:08.634629 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.635605 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.635611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:08.637477 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:08.637491 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:08.638330 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:08.638360 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:08.638618 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:08.639204 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:08.639230 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:08.641219 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:08.644316 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:08.644321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:08.644331 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:08.644701 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:08.645047 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:08.645050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:08.645052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:08.645212 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:08.645731 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:08.645736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:08.645995 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 18:52:08.646191 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 18:52:08.646558 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260810 18:52:08.648733 24471 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:08.649006 24471 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:08.649734 24471 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:08.651921 24150 ts_manager.cc:194] Registered new tserver with Master: f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481)
I20260810 18:52:08.653911 24150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:55557
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:52:08.693822 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:08.998958 24485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:08.999205 24485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:08.999293 24485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:09.006037 24485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:09.006134 24485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:09.006246 24485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:52:09.017223 24485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:09.019676 24485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:09.021159 24485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:09.033561 24494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:09.033756 24502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:09.033792 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:09.034359 24485 server_base.cc:1034] running on GCE node
I20260810 18:52:09.034926 24485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:09.035801 24485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:09.037004 24485 hybrid_clock.cc:648] HybridClock initialized: now 1786387929036945 us; error 53 us; skew 500 ppm
I20260810 18:52:09.037294 24485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:09 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:09.044685 24485 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:52:09.045928 24485 webserver.cc:460] Webserver started at http://127.29.174.194:33717/ using document root <none> and password file <none>
I20260810 18:52:09.046582 24485 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:09.046694 24485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:09.046970 24485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:09.049639 24485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "68972503abf440109bf8773513d2be46"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "b31606ea670ca398d3d5a150363eade7"
server_key_iv: "0e093466b375c04c1eecce5cebfc85d1"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.050482 24485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "68972503abf440109bf8773513d2be46"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "b31606ea670ca398d3d5a150363eade7"
server_key_iv: "0e093466b375c04c1eecce5cebfc85d1"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.055168 24485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:52:09.058267 24511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.059491 24485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:09.059633 24485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "68972503abf440109bf8773513d2be46"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "b31606ea670ca398d3d5a150363eade7"
server_key_iv: "0e093466b375c04c1eecce5cebfc85d1"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.059808 24485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:09.075275 24485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:09.079203 24485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:09.079438 24485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:09.080247 24485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:09.081408 24485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:09.081487 24485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.081621 24485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:09.081673 24485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.110795 24485 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:40411
I20260810 18:52:09.110854 24689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:40411 every 8 connection(s)
I20260810 18:52:09.112023 24485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:09.113276 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 24485
I20260810 18:52:09.113500 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:52:09.114321 30395 external_mini_cluster.cc:1428] Setting key 993c2cc04d2689b2f9ff8b7a1c1487cd
Aug 10 18:52:09 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:09.132422 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.114990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:54971 (local address 127.29.174.254:39349)
0810 18:52:09.115192 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.115197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.116333 (+ 1136us) server_negotiation.cc:407] Connection header received
0810 18:52:09.118529 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.118534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.118657 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.118755 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:09.120524 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.121480 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.122954 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.123507 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.123633 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.126696 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:09.126702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:09.126707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:09.126743 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:09.129642 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.130283 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.130289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.130292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.130461 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.130876 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.130881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.130885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.131137 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:09.131278 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.131872 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.132091 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:52:09.133183 24694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.115414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:54971)
0810 18:52:09.116179 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.116222 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.118233 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.119192 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.119207 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.119809 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:09.120372 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.120390 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.121605 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.121609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.122822 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.122835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.123687 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.123691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.123872 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.124442 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:09.124467 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:09.126486 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:09.129764 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:09.129772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:09.129786 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:09.130162 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:09.130576 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:09.130580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:09.130584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:09.130763 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:09.131253 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:09.131260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:09.131536 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.131758 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.132195 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":316,"threads_started":1}
I20260810 18:52:09.134461 24692 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:09.134769 24692 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:09.135531 24692 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:09.136969 24150 ts_manager.cc:194] Registered new tserver with Master: 68972503abf440109bf8773513d2be46 (127.29.174.194:40411)
I20260810 18:52:09.137704 24150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:54971
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:52:09.174376 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:09.481382 24702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:09.481611 24702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:09.481706 24702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:09.488483 24702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:09.488574 24702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:09.488682 24702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:52:09.497926 24702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:09.498924 24702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:09.500033 24702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:09.511878 24709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:09.511848 24713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:09.511915 24710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:09.512521 24702 server_base.cc:1034] running on GCE node
I20260810 18:52:09.513001 24702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:09.513800 24702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:09.514981 24702 hybrid_clock.cc:648] HybridClock initialized: now 1786387929514937 us; error 39 us; skew 500 ppm
I20260810 18:52:09.515273 24702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:09 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:09.518144 24702 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:52:09.519518 24702 webserver.cc:460] Webserver started at http://127.29.174.195:40673/ using document root <none> and password file <none>
I20260810 18:52:09.520107 24702 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:09.520217 24702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:09.520483 24702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:52:09.523198 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "d95c2761f05346d6b751b3eb9604deb8"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "bbe12c70f4b4a3e7fd40616f407e0dc8"
server_key_iv: "1f2319dd766339006a616a2ef8f30686"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.523895 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "d95c2761f05346d6b751b3eb9604deb8"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "bbe12c70f4b4a3e7fd40616f407e0dc8"
server_key_iv: "1f2319dd766339006a616a2ef8f30686"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.528390 24702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:52:09.531412 24723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.532536 24702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:09.532675 24702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "d95c2761f05346d6b751b3eb9604deb8"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "bbe12c70f4b4a3e7fd40616f407e0dc8"
server_key_iv: "1f2319dd766339006a616a2ef8f30686"
server_key_version: "encryptionkey@0"
I20260810 18:52:09.532832 24702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:09.558406 24702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:09.561798 24702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:09.562036 24702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:09.562923 24702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:09.564138 24702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:09.564220 24702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.564327 24702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:09.564378 24702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:09.593127 24702 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37503
I20260810 18:52:09.593189 24908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37503 every 8 connection(s)
I20260810 18:52:09.594290 24702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:09.594640 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 24702
I20260810 18:52:09.594874 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:52:09.595453 30395 external_mini_cluster.cc:1428] Setting key 91cb065ade9e89cdd76a4b456a5427e2
Aug 10 18:52:09 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387929, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:09.612377 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.596887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56883 (local address 127.29.174.254:39349)
0810 18:52:09.597073 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.597079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.598276 (+ 1197us) server_negotiation.cc:407] Connection header received
0810 18:52:09.600046 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.600050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.600116 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.600192 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:09.601483 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.602210 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.603743 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.604166 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.604266 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.607295 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:09.607300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:09.607303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:09.607335 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:09.609852 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.610671 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.610675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.610677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.610805 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.611123 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.611126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.611128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.611378 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:09.611481 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.611979 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.612139 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:52:09.613165 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.597381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:56883)
0810 18:52:09.598133 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.598175 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.599823 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.600364 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.600372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.600835 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:09.601361 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.601377 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.602331 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.602335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.603621 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.603638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.604378 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.604383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.604585 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.605117 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:09.605137 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:09.607116 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:09.609997 (+ 2881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:09.610005 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:09.610017 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:09.610568 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:09.610881 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:09.610884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:09.610886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:09.611047 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:09.611465 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:09.611469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:09.611713 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.611911 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.612234 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":315,"threads_started":1}
I20260810 18:52:09.614369 24909 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:09.614629 24909 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:09.615239 24909 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:09.616513 24150 ts_manager.cc:194] Registered new tserver with Master: d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503)
I20260810 18:52:09.617259 24150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:56883
I20260810 18:52:09.622207 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:09.638616 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.626446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.29.174.254:39349)
0810 18:52:09.626601 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.626606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.627048 (+ 442us) server_negotiation.cc:407] Connection header received
0810 18:52:09.627195 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.627198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.627265 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.627336 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:09.629108 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.629869 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.631730 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.632118 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.632203 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.634251 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:09.634256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:09.634259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:09.634285 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:09.636426 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.637068 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.637072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.637074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.637193 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:09.637496 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:09.637499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:09.637501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:09.637770 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:09.637945 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.638166 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.638322 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:52:09.641899 24150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:09.648425 24150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:09.656612 24471 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:09.683267 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.659413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.29.174.193:35481)
0810 18:52:09.675521 (+ 16108us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.675530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.675558 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:52:09.675644 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.675651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.675866 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.676044 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:09.677052 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.678748 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.681243 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.682126 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.682316 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.682516 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.682804 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.682965 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15942,"thread_start_us":16808,"threads_started":1}
I20260810 18:52:09.683288 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.659216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35481 (local address 127.0.0.1:38010)
0810 18:52:09.671482 (+ 12266us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.671521 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.671684 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.676061 (+ 4377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.676068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.676107 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:09.676457 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.676469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.678882 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.678886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.681088 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.681103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.682330 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.682336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.682561 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.682616 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.682752 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.682939 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12058,"thread_start_us":17776,"threads_started":1}
I20260810 18:52:09.685580 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.659345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37503 (local address 127.0.0.1:39966)
0810 18:52:09.677243 (+ 17898us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.677257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.677353 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.678289 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.678294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.678318 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:09.678759 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.678770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.680338 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.680343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.682483 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.682499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.684842 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.684848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.685002 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.685033 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.685108 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.685228 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17778,"mutex_wait_us":6,"thread_start_us":11796,"threads_started":1}
I20260810 18:52:09.685628 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.659890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.29.174.195:37503)
0810 18:52:09.676664 (+ 16774us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.676673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.677355 (+ 682us) server_negotiation.cc:407] Connection header received
0810 18:52:09.677642 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.677649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.677854 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.678009 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:09.678956 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.680178 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.682639 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.684662 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.684831 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.684977 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.685177 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.685329 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16610,"thread_start_us":16935,"threads_started":1}
I20260810 18:52:09.686026 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.660126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.29.174.194:40411)
0810 18:52:09.676779 (+ 16653us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.676788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.677757 (+ 969us) server_negotiation.cc:407] Connection header received
0810 18:52:09.678039 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.678046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.678250 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.678456 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:09.679183 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.680720 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.683395 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.684710 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.684865 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.685018 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.685565 (+ 547us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.685716 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16513,"thread_start_us":16379,"threads_started":1}
I20260810 18:52:09.686739 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.659711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:40411 (local address 127.0.0.1:59024)
0810 18:52:09.677644 (+ 17933us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.677660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.677859 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.678440 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.678445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.678473 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:09.679012 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.679031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.680865 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.680870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.682917 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.682930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.685170 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.685176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.685355 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.685387 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.686277 (+ 890us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.686419 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17814,"mutex_wait_us":21,"thread_start_us":17114,"threads_started":1}
I20260810 18:52:09.687196 24368 tablet_service.cc:1467] Processing CreateTablet for tablet 16d6e23414184ebba06fe7b524a84377 (DEFAULT_TABLE table=test-table [id=c30be47e7900435ba133ea5c03596b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:09.688656 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d6e23414184ebba06fe7b524a84377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:09.689103 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 16d6e23414184ebba06fe7b524a84377 (DEFAULT_TABLE table=test-table [id=c30be47e7900435ba133ea5c03596b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:09.690150 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d6e23414184ebba06fe7b524a84377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:09.690191 24591 tablet_service.cc:1467] Processing CreateTablet for tablet 16d6e23414184ebba06fe7b524a84377 (DEFAULT_TABLE table=test-table [id=c30be47e7900435ba133ea5c03596b88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:09.691431 24591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d6e23414184ebba06fe7b524a84377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:09.699918 24945 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Bootstrap starting.
I20260810 18:52:09.702410 24948 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Bootstrap starting.
I20260810 18:52:09.703469 24945 tablet_bootstrap.cc:654] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:09.704676 24945 log.cc:824] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:09.705066 24949 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Bootstrap starting.
I20260810 18:52:09.705927 24948 tablet_bootstrap.cc:654] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:09.707056 24948 log.cc:824] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:09.708071 24945 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: No bootstrap required, opened a new log
I20260810 18:52:09.708310 24949 tablet_bootstrap.cc:654] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:09.708369 24945 ts_tablet_manager.cc:1397] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 18:52:09.709468 24949 log.cc:824] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:09.709661 24948 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: No bootstrap required, opened a new log
I20260810 18:52:09.709928 24948 ts_tablet_manager.cc:1397] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260810 18:52:09.712527 24949 tablet_bootstrap.cc:492] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: No bootstrap required, opened a new log
I20260810 18:52:09.712785 24949 ts_tablet_manager.cc:1397] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260810 18:52:09.716985 24948 raft_consensus.cc:348] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.717411 24948 raft_consensus.cc:374] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:09.717514 24948 raft_consensus.cc:729] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a8e5c75c484867be80f4cb36c02207, State: Initialized, Role: FOLLOWER
I20260810 18:52:09.718070 24948 consensus_queue.cc:260] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.718093 24945 raft_consensus.cc:348] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.718511 24945 raft_consensus.cc:374] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:09.718606 24945 raft_consensus.cc:729] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95c2761f05346d6b751b3eb9604deb8, State: Initialized, Role: FOLLOWER
I20260810 18:52:09.719137 24945 consensus_queue.cc:260] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.720211 24909 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:09.721992 24945 ts_tablet_manager.cc:1428] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260810 18:52:09.722442 24949 raft_consensus.cc:348] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.722738 24949 raft_consensus.cc:374] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:09.722831 24949 raft_consensus.cc:729] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68972503abf440109bf8773513d2be46, State: Initialized, Role: FOLLOWER
I20260810 18:52:09.723318 24949 consensus_queue.cc:260] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.724337 24692 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:09.724941 24949 ts_tablet_manager.cc:1428] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 18:52:09.727080 24948 ts_tablet_manager.cc:1428] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
W20260810 18:52:09.845868 24910 tablet.cc:2369] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:09.867758 24693 tablet.cc:2369] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:09.880053 24473 tablet.cc:2369] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:09.988034 24955 raft_consensus.cc:482] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:09.988322 24955 raft_consensus.cc:504] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:09.990022 24955 leader_election.cc:288] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481), 68972503abf440109bf8773513d2be46 (127.29.174.194:40411)
I20260810 18:52:09.996145 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.990246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35481 (local address 127.29.174.195:47127)
0810 18:52:09.990485 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.990505 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.990648 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.991167 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.991172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.991207 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:09.991415 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.991428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.992595 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.992599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.994417 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.994433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.995647 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.995651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.995780 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.995806 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.995873 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.995960 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:52:09.996268 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.990305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:47127 (local address 127.29.174.193:35481)
0810 18:52:09.990521 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.990526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.990574 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:52:09.990795 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.990801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.990887 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.990982 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:09.991561 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.992459 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.994558 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.995505 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.995651 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.995737 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.995912 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.996036 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:52:09.997150 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.990528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:40411 (local address 127.29.174.195:42373)
0810 18:52:09.991404 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:09.991421 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:09.991541 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:09.991842 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:09.991845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:09.991862 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:09.992207 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.992219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.993222 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.993228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.994947 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:09.994958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.996488 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.996494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:09.996613 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.996636 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:52:09.996716 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:52:09.996840 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":525,"threads_started":1}
I20260810 18:52:09.997143 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:09.990863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:42373 (local address 127.29.174.194:40411)
0810 18:52:09.991062 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:09.991068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:09.991515 (+ 447us) server_negotiation.cc:407] Connection header received
0810 18:52:09.991658 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:09.991663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:09.991739 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:09.991829 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:09.992353 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.993087 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.995101 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:09.996202 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:09.996364 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:09.996429 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:09.996787 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 18:52:09.996911 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:52:09.997100 24401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d6e23414184ebba06fe7b524a84377" candidate_uuid: "d95c2761f05346d6b751b3eb9604deb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a8e5c75c484867be80f4cb36c02207" is_pre_election: true
I20260810 18:52:09.997601 24401 raft_consensus.cc:2454] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95c2761f05346d6b751b3eb9604deb8 in term 0.
I20260810 18:52:09.998224 24726 leader_election.cc:302] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d95c2761f05346d6b751b3eb9604deb8, f2a8e5c75c484867be80f4cb36c02207; no voters:
I20260810 18:52:09.998621 24955 raft_consensus.cc:2781] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:09.998739 24955 raft_consensus.cc:482] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:09.998833 24955 raft_consensus.cc:3037] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:09.998703 24621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d6e23414184ebba06fe7b524a84377" candidate_uuid: "d95c2761f05346d6b751b3eb9604deb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68972503abf440109bf8773513d2be46" is_pre_election: true
I20260810 18:52:09.999157 24621 raft_consensus.cc:2454] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d95c2761f05346d6b751b3eb9604deb8 in term 0.
I20260810 18:52:10.000991 24955 raft_consensus.cc:504] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:10.001590 24955 leader_election.cc:288] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [CANDIDATE]: Term 1 election: Requested vote from peers f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481), 68972503abf440109bf8773513d2be46 (127.29.174.194:40411)
I20260810 18:52:10.001930 24401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d6e23414184ebba06fe7b524a84377" candidate_uuid: "d95c2761f05346d6b751b3eb9604deb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a8e5c75c484867be80f4cb36c02207"
I20260810 18:52:10.001997 24621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d6e23414184ebba06fe7b524a84377" candidate_uuid: "d95c2761f05346d6b751b3eb9604deb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68972503abf440109bf8773513d2be46"
I20260810 18:52:10.002097 24401 raft_consensus.cc:3037] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:10.002166 24621 raft_consensus.cc:3037] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:10.004240 24401 raft_consensus.cc:2454] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95c2761f05346d6b751b3eb9604deb8 in term 1.
I20260810 18:52:10.004295 24621 raft_consensus.cc:2454] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d95c2761f05346d6b751b3eb9604deb8 in term 1.
I20260810 18:52:10.004576 24726 leader_election.cc:302] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d95c2761f05346d6b751b3eb9604deb8, f2a8e5c75c484867be80f4cb36c02207; no voters:
I20260810 18:52:10.004845 24955 raft_consensus.cc:2781] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:10.005267 24955 raft_consensus.cc:686] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 1 LEADER]: Becoming Leader. State: Replica: d95c2761f05346d6b751b3eb9604deb8, State: Running, Role: LEADER
I20260810 18:52:10.005679 24955 consensus_queue.cc:237] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:10.010187 24150 catalog_manager.cc:5495] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "d95c2761f05346d6b751b3eb9604deb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260810 18:52:10.056645 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.051541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.29.174.195:37503)
0810 18:52:10.051705 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:10.051712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:10.051807 (+ 95us) server_negotiation.cc:407] Connection header received
0810 18:52:10.052022 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:10.052026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:10.052102 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:10.052236 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:10.052596 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.053283 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.054693 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.055084 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.055192 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:10.055658 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:10.056112 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:10.056246 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:10.056339 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 18:52:10.056418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:52:10.064630 24401 raft_consensus.cc:1261] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Refusing update from remote peer d95c2761f05346d6b751b3eb9604deb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:10.064636 24621 raft_consensus.cc:1261] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Refusing update from remote peer d95c2761f05346d6b751b3eb9604deb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:10.065531 24955 consensus_queue.cc:1035] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:10.065814 24959 consensus_queue.cc:1035] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:10.075090 24962 mvcc.cc:205] Tried to move back new op lower bound from 7317044961534550016 to 7317044961306750976. Current Snapshot: MvccSnapshot[applied={T|T < 7317044961534550016}]
I20260810 18:52:10.076447 24964 mvcc.cc:205] Tried to move back new op lower bound from 7317044961534550016 to 7317044961306750976. Current Snapshot: MvccSnapshot[applied={T|T < 7317044961534550016}]
I20260810 18:52:10.081148 24963 mvcc.cc:205] Tried to move back new op lower bound from 7317044961534550016 to 7317044961306750976. Current Snapshot: MvccSnapshot[applied={T|T < 7317044961534550016}]
I20260810 18:52:10.087661 24149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:10.087913 24149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:10.091221 24149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Sending DeleteTablet for 3 replicas of tablet 16d6e23414184ebba06fe7b524a84377
I20260810 18:52:10.092181 24368 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d6e23414184ebba06fe7b524a84377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:10 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:38010
I20260810 18:52:10.092236 24799 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d6e23414184ebba06fe7b524a84377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:10 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:39966
I20260810 18:52:10.092519 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 24239
I20260810 18:52:10.092914 24972 tablet_replica.cc:330] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: stopping tablet replica
I20260810 18:52:10.092878 24591 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d6e23414184ebba06fe7b524a84377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:10 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:59024
I20260810 18:52:10.093330 24972 raft_consensus.cc:2229] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:10.093640 24975 tablet_replica.cc:330] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: stopping tablet replica
I20260810 18:52:10.093827 24972 raft_consensus.cc:2258] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:10.094045 24975 raft_consensus.cc:2229] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:10.094552 24975 raft_consensus.cc:2258] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:10.095821 24972 ts_tablet_manager.cc:1905] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:10.096336 24975 ts_tablet_manager.cc:1905] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:10.102918 24972 ts_tablet_manager.cc:1918] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:10.103112 24972 log.cc:1196] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/16d6e23414184ebba06fe7b524a84377
I20260810 18:52:10.103121 24975 ts_tablet_manager.cc:1918] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:10.103296 24975 log.cc:1196] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/16d6e23414184ebba06fe7b524a84377
I20260810 18:52:10.103639 24972 ts_tablet_manager.cc:1939] T 16d6e23414184ebba06fe7b524a84377 P f2a8e5c75c484867be80f4cb36c02207: Deleting consensus metadata
I20260810 18:52:10.103837 24975 ts_tablet_manager.cc:1939] T 16d6e23414184ebba06fe7b524a84377 P 68972503abf440109bf8773513d2be46: Deleting consensus metadata
I20260810 18:52:10.105370 24127 catalog_manager.cc:4841] TS f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481): tablet 16d6e23414184ebba06fe7b524a84377 (table test-table [id=c30be47e7900435ba133ea5c03596b88]) successfully deleted
I20260810 18:52:10.105459 24130 catalog_manager.cc:4841] TS 68972503abf440109bf8773513d2be46 (127.29.174.194:40411): tablet 16d6e23414184ebba06fe7b524a84377 (table test-table [id=c30be47e7900435ba133ea5c03596b88]) successfully deleted
I20260810 18:52:10.105882 24974 tablet_replica.cc:330] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: stopping tablet replica
I20260810 18:52:10.106168 24974 raft_consensus.cc:2229] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:10.106645 24974 raft_consensus.cc:2258] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:10.108472 24974 ts_tablet_manager.cc:1905] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:10.115746 24974 ts_tablet_manager.cc:1918] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:10.115962 24974 log.cc:1196] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/16d6e23414184ebba06fe7b524a84377
I20260810 18:52:10.116474 24974 ts_tablet_manager.cc:1939] T 16d6e23414184ebba06fe7b524a84377 P d95c2761f05346d6b751b3eb9604deb8: Deleting consensus metadata
I20260810 18:52:10.118116 24125 catalog_manager.cc:4841] TS d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503): tablet 16d6e23414184ebba06fe7b524a84377 (table test-table [id=c30be47e7900435ba133ea5c03596b88]) successfully deleted
Aug 10 18:52:10 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:52:10.127645 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.093556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.29.174.193:35481)
0810 18:52:10.093735 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:10.093740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:10.107677 (+ 13937us) server_negotiation.cc:407] Connection header received
0810 18:52:10.107894 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:10.107899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:10.107975 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:10.108079 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:10.109870 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.110992 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.113113 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.113741 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.113841 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:10.120643 (+ 6802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:10.120660 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:10.120677 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:10.120719 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:10.124534 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.125342 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.125352 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.125356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.125514 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.125898 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.125904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.125907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.126390 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:10.126700 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:10.127056 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 18:52:10.127275 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:52:10.128264 24465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:10.214831 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 24239
I20260810 18:52:10.249300 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 24485
Aug 10 18:52:10 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:52:10.264887 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.250063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.29.174.194:40411)
0810 18:52:10.250255 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:10.250261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:10.250280 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:52:10.250516 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:10.250521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:10.250588 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:10.250680 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:10.251933 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.252630 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.253952 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.254383 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.254496 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:10.258516 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:10.258535 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:10.258554 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:10.258595 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:10.262056 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.262769 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.262779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.262783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.262937 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.263299 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.263304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.263307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.263764 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:10.264063 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:10.264280 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 18:52:10.264518 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:52:10.265615 24682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:10.351323 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 24485
I20260810 18:52:10.374642 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 24702
Aug 10 18:52:10 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:52:10.390233 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:10.375350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.29.174.195:37503)
0810 18:52:10.375519 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:10.375525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:10.375613 (+ 88us) server_negotiation.cc:407] Connection header received
0810 18:52:10.375798 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:10.375807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:10.375887 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:10.375981 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:10.377300 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.378088 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.379457 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:10.379912 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:10.380023 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:10.384351 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:10.384362 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:10.384376 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:10.384408 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:10.387449 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.388137 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.388144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.388147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.388332 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:10.388680 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:10.388683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:10.388685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:10.389079 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:10.389401 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:10.389647 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 18:52:10.389855 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:52:10.390910 24899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:10.508834 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 24702
I20260810 18:52:10.532922 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 24094
I20260810 18:52:10.533979 24193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:10.642661 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 24094
I20260810 18:52:30.665315 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--webserver_interface=127.29.174.254
--webserver_port=45747
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:30.987963 24987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:30.988225 24987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:30.988359 24987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:30.995266 24987 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:52:30.995357 24987 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:52:30.995421 24987 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:52:30.995496 24987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:30.995569 24987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:30.995661 24987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:30.995720 24987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:30.995795 24987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:31.004998 24987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=45747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:31.006071 24987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:31.007268 24987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:31.029775 24997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:31.030290 24987 server_base.cc:1034] running on GCE node
W20260810 18:52:31.034474 25002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.034727 24998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:31.035631 24987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:31.037509 24987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:31.038730 24987 hybrid_clock.cc:648] HybridClock initialized: now 1786387951038665 us; error 36 us; skew 500 ppm
I20260810 18:52:31.039067 24987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:31 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:31.047678 24987 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:52:31.049185 24987 webserver.cc:460] Webserver started at http://127.29.174.254:45747/ using document root <none> and password file <none>
I20260810 18:52:31.049924 24987 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:31.050045 24987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:31.055575 24987 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:31.058390 25012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:31.059602 24987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:31.059762 24987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "462e0e7ccb8248f8a51cd7589cdf4554"
format_stamp: "Formatted at 2026-08-10 18:52:07 on dist-test-slave-6zbq"
server_key: "d937507dd69fe42d6cd3de10767a9793"
server_key_iv: "6abea1c51e9a6f28c355688275313b8f"
server_key_version: "encryptionkey@0"
I20260810 18:52:31.060721 24987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:31.075822 24987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:31.079468 24987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:31.079735 24987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:31.097108 24987 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39349
I20260810 18:52:31.097131 25111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39349 every 8 connection(s)
I20260810 18:52:31.098505 24987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:31.101701 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 24987
I20260810 18:52:31.102535 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:35481
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=40319
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:31.105086 25112 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:52:31.108196 25112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap starting.
I20260810 18:52:31.130983 25112 log.cc:824] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:31.146664 25112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:52:31.147295 25112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap complete.
I20260810 18:52:31.160482 25112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:31.161648 25112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Initialized, Role: FOLLOWER
I20260810 18:52:31.162335 25112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:31.162659 25112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:31.162786 25112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:31.162961 25112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:52:31.168556 25112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:31.169204 25112 leader_election.cc:302] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462e0e7ccb8248f8a51cd7589cdf4554; no voters:
I20260810 18:52:31.169860 25112 leader_election.cc:288] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:52:31.170003 25116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:52:31.171146 25116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 LEADER]: Becoming Leader. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Running, Role: LEADER
I20260810 18:52:31.171679 25116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:31.172017 25112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:31.174003 25118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462e0e7ccb8248f8a51cd7589cdf4554. Latest consensus state: current_term: 2 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:31.174027 25117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:31.174475 25118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:31.174472 25117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:31.183347 25123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:31.190563 25123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:31.191982 25123 catalog_manager.cc:1259] Loaded cluster ID: df7d73688feb45378cc62205b99971d1
I20260810 18:52:31.192114 25123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:31.197487 25123 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:31.199102 25123 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Loaded TSK: 0
I20260810 18:52:31.233072 25123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Generated new TSK 1
I20260810 18:52:31.233455 25123 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:52:31.469064 25114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:31.469312 25114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:31.469424 25114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:31.476433 25114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:31.476523 25114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:31.476652 25114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:52:31.485725 25114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:35481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=40319
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:31.486815 25114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:31.487962 25114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:31.499917 25156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.500082 25154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.500499 25152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:31.500959 25114 server_base.cc:1034] running on GCE node
I20260810 18:52:31.501530 25114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:31.502456 25114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:31.503651 25114 hybrid_clock.cc:648] HybridClock initialized: now 1786387951503590 us; error 55 us; skew 500 ppm
I20260810 18:52:31.503927 25114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:31 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:31.506815 25114 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:52:31.508176 25114 webserver.cc:460] Webserver started at http://127.29.174.193:40319/ using document root <none> and password file <none>
I20260810 18:52:31.508774 25114 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:31.508893 25114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:31.514156 25114 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:31.516676 25167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:31.517897 25114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:52:31.518036 25114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "f2a8e5c75c484867be80f4cb36c02207"
format_stamp: "Formatted at 2026-08-10 18:52:08 on dist-test-slave-6zbq"
server_key: "107fb834a6b20502ab1ee141ff75f10a"
server_key_iv: "a48d2dd65cbf017717ef79b833cb3d0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:31.518985 25114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:31.538035 25114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:31.541986 25114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:31.542217 25114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:31.543121 25114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:31.544353 25114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:31.544440 25114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:31.544543 25114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:31.544617 25114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:31.578424 25114 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35481
I20260810 18:52:31.578509 25350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35481 every 8 connection(s)
I20260810 18:52:31.579721 25114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:31.585999 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 25114
I20260810 18:52:31.586793 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:40411
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=33717
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:52:31 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:31.601378 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:31.582720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:47533 (local address 127.29.174.254:39349)
0810 18:52:31.583339 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:31.583356 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:31.583773 (+ 417us) server_negotiation.cc:407] Connection header received
0810 18:52:31.585749 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:31.585761 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:31.586044 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:31.586413 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:31.587956 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.589131 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.590649 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.591261 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.591451 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:31.594643 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:31.594650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:31.594663 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:31.594704 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:31.597291 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:31.597959 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:31.597967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:31.597974 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:31.598133 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:31.598593 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:31.598598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:31.598600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:31.599015 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:31.599187 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:31.599893 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 18:52:31.600254 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":236,"threads_started":1}
I20260810 18:52:31.601409 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:31.582872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:47533)
0810 18:52:31.583617 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:31.583653 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:31.585253 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:31.586569 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:31.586579 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:31.587266 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:31.587751 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:31.587776 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.589295 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.589301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:31.590515 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:31.590528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:31.591500 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:31.591508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:31.591759 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:31.592338 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:31.592364 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:31.594391 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:31.597420 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:31.597428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:31.597441 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:31.597827 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:31.598240 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:31.598245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:31.598249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:31.598481 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:31.599148 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:31.599156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:31.599451 (+ 295us) client_negotiation.cc:769] Sending connection context
0810 18:52:31.599664 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 18:52:31.600152 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":293,"threads_started":1}
I20260810 18:52:31.604453 25351 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:31.604718 25351 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:31.605371 25351 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:31.607364 25044 ts_manager.cc:194] Registered new tserver with Master: f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481)
I20260810 18:52:31.608799 25044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:47533
W20260810 18:52:31.961496 25357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:31.961740 25357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:31.961840 25357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:31.968976 25357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:31.969076 25357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:31.969193 25357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:52:31.978628 25357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:40411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=33717
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:31.979763 25357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:31.980971 25357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:31.993220 25373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.993266 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:31.993287 25370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:31.994405 25357 server_base.cc:1034] running on GCE node
I20260810 18:52:31.994977 25357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:31.995805 25357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:31.996984 25357 hybrid_clock.cc:648] HybridClock initialized: now 1786387951996935 us; error 43 us; skew 500 ppm
I20260810 18:52:31.997259 25357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:31 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:32.000164 25357 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:52:32.001526 25357 webserver.cc:460] Webserver started at http://127.29.174.194:33717/ using document root <none> and password file <none>
I20260810 18:52:32.002135 25357 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:32.002261 25357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:32.007160 25357 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:52:32.009647 25387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.010890 25357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:32.011030 25357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "68972503abf440109bf8773513d2be46"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "b31606ea670ca398d3d5a150363eade7"
server_key_iv: "0e093466b375c04c1eecce5cebfc85d1"
server_key_version: "encryptionkey@0"
I20260810 18:52:32.011921 25357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:32.035174 25357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:32.038630 25357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:32.038847 25357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:32.039742 25357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:32.040953 25357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:32.041041 25357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.041159 25357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:32.041237 25357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.071630 25357 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:40411
I20260810 18:52:32.071702 25573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:40411 every 8 connection(s)
I20260810 18:52:32.073073 25357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:32.082463 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 25357
I20260810 18:52:32.083078 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:37503
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=40673
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:52:32 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387951, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:32.091624 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.075307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:38723 (local address 127.29.174.254:39349)
0810 18:52:32.075508 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.075514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.076543 (+ 1029us) server_negotiation.cc:407] Connection header received
0810 18:52:32.078260 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.078265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.078332 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.078445 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.079746 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.080415 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.081854 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.082455 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.082583 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.085923 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.085929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.085933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:32.085970 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.088785 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.089484 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.089490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.089493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.089632 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.089990 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.089993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.089996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.090319 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.090468 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.091128 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.091335 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:52:32.092382 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.075733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:38723)
0810 18:52:32.076386 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.076430 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.078027 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.078608 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.078615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.079100 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:32.079617 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.079634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.080535 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.080538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.081732 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.081745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.082663 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.082669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.082868 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.083534 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:32.083564 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:32.085712 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:32.088922 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:32.088928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:32.088938 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:32.089371 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:32.089723 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:32.089726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:32.089729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:32.089897 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:32.090482 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:32.090488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:32.090788 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.091011 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.091370 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260810 18:52:32.093681 25574 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:32.093966 25574 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:32.094741 25574 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:32.096068 25044 ts_manager.cc:194] Registered new tserver with Master: 68972503abf440109bf8773513d2be46 (127.29.174.194:40411)
I20260810 18:52:32.097133 25044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:38723
W20260810 18:52:32.442400 25579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:32.442667 25579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:32.442791 25579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:32.449666 25579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:32.449774 25579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:32.449930 25579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:52:32.459076 25579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:37503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=40673
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:32.460326 25579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:32.461604 25579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:32.473922 25589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:32.473862 25590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:32.473892 25596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:32.474649 25579 server_base.cc:1034] running on GCE node
I20260810 18:52:32.475209 25579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:32.476055 25579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:32.477252 25579 hybrid_clock.cc:648] HybridClock initialized: now 1786387952477209 us; error 37 us; skew 500 ppm
I20260810 18:52:32.477532 25579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:32 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387952, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:32.480413 25579 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:52:32.481693 25579 webserver.cc:460] Webserver started at http://127.29.174.195:40673/ using document root <none> and password file <none>
I20260810 18:52:32.482300 25579 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:32.482456 25579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:32.487349 25579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:52:32.489753 25604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.490917 25579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:52:32.491068 25579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "d95c2761f05346d6b751b3eb9604deb8"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "bbe12c70f4b4a3e7fd40616f407e0dc8"
server_key_iv: "1f2319dd766339006a616a2ef8f30686"
server_key_version: "encryptionkey@0"
I20260810 18:52:32.491974 25579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:32.509267 25579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:32.513633 25579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:32.513903 25579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:32.514973 25579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:32.516319 25579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:32.516415 25579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.516515 25579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:32.516584 25579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:32.547405 25579 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37503
I20260810 18:52:32.547461 25792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37503 every 8 connection(s)
I20260810 18:52:32.548770 25579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:32.555441 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 25579
Aug 10 18:52:32 dist-test-slave-6zbq krb5kdc[24076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387952, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:52:32.569484 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.551133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:50955 (local address 127.29.174.254:39349)
0810 18:52:32.551310 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.551329 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.552474 (+ 1145us) server_negotiation.cc:407] Connection header received
0810 18:52:32.554219 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.554225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.554303 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.554427 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.555929 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.556924 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.558998 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.559442 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.559583 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.563733 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.563739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.563744 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:32.563782 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.566490 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.567174 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.567179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.567183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.567358 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.567773 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.567777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.567779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.568054 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.568230 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.568894 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.569117 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:52:32.570482 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.551636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:50955)
0810 18:52:32.552316 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.552359 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.553963 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.554650 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.554657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.555173 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:52:32.555748 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.555768 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.557112 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.557118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.558809 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.558826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.559711 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.559718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.559996 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.560597 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:52:32.560618 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:52:32.563529 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:52:32.566619 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:32.566626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:32.566637 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:32.567057 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:32.567467 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:52:32.567469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:52:32.567472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:52:32.567651 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:52:32.568184 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:52:32.568191 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:52:32.568535 (+ 344us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.568772 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.569298 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":258,"threads_started":1}
I20260810 18:52:32.571528 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.556427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.29.174.254:39349)
0810 18:52:32.557492 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.557497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.557512 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:52:32.557553 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.557557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.557614 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.557701 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.559854 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.560569 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.562533 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.563037 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.563134 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.566099 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.566107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.566112 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:32.566162 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.568924 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.569559 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.569564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.569568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.569701 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.570054 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.570059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.570063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.570324 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.570597 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.570948 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.571149 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"spinlock_wait_cycles":1920,"thread_start_us":863,"threads_started":1}
I20260810 18:52:32.572026 25793 heartbeater.cc:347] Connected to a master server at 127.29.174.254:39349
I20260810 18:52:32.572362 25793 heartbeater.cc:464] Registering TS with master...
I20260810 18:52:32.573241 25793 heartbeater.cc:511] Master 127.29.174.254:39349 requested a full tablet report, sending...
I20260810 18:52:32.574712 25044 ts_manager.cc:194] Registered new tserver with Master: d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503)
I20260810 18:52:32.575460 25044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:50955
I20260810 18:52:32.585896 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:52:32.591652 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.586735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34046 (local address 127.29.174.254:39349)
0810 18:52:32.586942 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.586949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.587336 (+ 387us) server_negotiation.cc:407] Connection header received
0810 18:52:32.587486 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.587490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.587604 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.587715 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:32.588036 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.588684 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.590058 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.590489 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.590578 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.590989 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:32.591248 (+ 259us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 18:52:32.591489 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34046: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 18:52:32.591765 25799 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34046: token expired
W20260810 18:52:32.592330 24912 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.174.254:39349)
I20260810 18:52:32.604755 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.592890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.29.174.254:39349)
0810 18:52:32.593056 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.593062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.593075 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:52:32.593108 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.593112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.593161 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.593238 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.594675 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.595385 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.596667 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.597020 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.597132 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.599648 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.599652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.599655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:32.599685 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.601977 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.602735 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.602740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.602744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.602864 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.603216 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.603221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.603224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.603625 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.603857 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.604204 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.604416 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:52:32.611794 25351 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:32.608541 25044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:52:32.617166 25044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:52:32.639580 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.629655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.29.174.193:35481)
0810 18:52:32.631349 (+ 1694us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.631358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.631387 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:52:32.631485 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.631492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.631750 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.631940 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:32.632738 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.634525 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.637171 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.638361 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.638550 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.638730 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.639076 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.639244 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1401,"threads_started":1}
I20260810 18:52:32.639617 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.628537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35481 (local address 127.0.0.1:41156)
0810 18:52:32.629595 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.629638 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.629799 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.632023 (+ 2224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.632032 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.632068 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:32.632522 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.632534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.634703 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.634708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.636975 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.636992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.638600 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.638607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.638829 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.638889 (+ 60us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.639013 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.639177 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":19,"thread_start_us":6029,"threads_started":1}
I20260810 18:52:32.640736 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.628812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37503 (local address 127.0.0.1:39778)
0810 18:52:32.633064 (+ 4252us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.633081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.633278 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.634163 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.634167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.634189 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:32.634575 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.634591 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.636306 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.636312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.638199 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.638213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.640078 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.640085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.640217 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.640240 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.640322 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.640439 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4132,"thread_start_us":372,"threads_started":1}
I20260810 18:52:32.640954 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.628774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.29.174.195:37503)
0810 18:52:32.632597 (+ 3823us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.632606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.633171 (+ 565us) server_negotiation.cc:407] Connection header received
0810 18:52:32.633467 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.633475 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.633691 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.633857 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:32.634848 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.636181 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.638379 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.639582 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.639766 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.639929 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.640389 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.640537 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3664,"thread_start_us":3508,"threads_started":1}
I20260810 18:52:32.643435 25244 tablet_service.cc:1467] Processing CreateTablet for tablet 879e7313e84a478c8171869e3cc63bea (DEFAULT_TABLE table=test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:32.644047 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.629335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:40411 (local address 127.0.0.1:43350)
0810 18:52:32.634877 (+ 5542us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.634897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.635021 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.635575 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.635579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.635609 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:32.636007 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.636018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.637757 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.637763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.639791 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.639805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.643314 (+ 3509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.643321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.643482 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.643514 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.643612 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.643730 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5394,"mutex_wait_us":1043,"thread_start_us":2406,"threads_started":1}
I20260810 18:52:32.644132 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.630198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.29.174.194:40411)
0810 18:52:32.630874 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.630884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.635012 (+ 4128us) server_negotiation.cc:407] Connection header received
0810 18:52:32.635159 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.635167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.635375 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.635562 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:32.636169 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.637620 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.639959 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.643080 (+ 3121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.643256 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.643418 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.643663 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.643827 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":370,"threads_started":1}
I20260810 18:52:32.644184 25678 tablet_service.cc:1467] Processing CreateTablet for tablet 879e7313e84a478c8171869e3cc63bea (DEFAULT_TABLE table=test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:32.645328 25678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879e7313e84a478c8171869e3cc63bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:32.645715 25244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879e7313e84a478c8171869e3cc63bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:32.647328 25460 tablet_service.cc:1467] Processing CreateTablet for tablet 879e7313e84a478c8171869e3cc63bea (DEFAULT_TABLE table=test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:52:32.648563 25460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 879e7313e84a478c8171869e3cc63bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:52:32.657269 25814 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Bootstrap starting.
I20260810 18:52:32.660748 25815 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Bootstrap starting.
I20260810 18:52:32.661047 25814 tablet_bootstrap.cc:654] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:32.662443 25814 log.cc:824] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:32.662817 25817 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Bootstrap starting.
I20260810 18:52:32.664291 25815 tablet_bootstrap.cc:654] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:32.665553 25815 log.cc:824] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:32.665935 25817 tablet_bootstrap.cc:654] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Neither blocks nor log segments found. Creating new log.
I20260810 18:52:32.667078 25817 log.cc:824] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:32.668696 25815 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: No bootstrap required, opened a new log
I20260810 18:52:32.668977 25815 ts_tablet_manager.cc:1397] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260810 18:52:32.670233 25814 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: No bootstrap required, opened a new log
I20260810 18:52:32.670533 25814 ts_tablet_manager.cc:1397] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260810 18:52:32.670866 25817 tablet_bootstrap.cc:492] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: No bootstrap required, opened a new log
I20260810 18:52:32.671113 25817 ts_tablet_manager.cc:1397] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260810 18:52:32.675879 25815 raft_consensus.cc:348] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.676210 25815 raft_consensus.cc:374] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:32.676306 25815 raft_consensus.cc:729] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a8e5c75c484867be80f4cb36c02207, State: Initialized, Role: FOLLOWER
I20260810 18:52:32.676858 25815 consensus_queue.cc:260] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.678642 25815 ts_tablet_manager.cc:1428] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 18:52:32.680589 25814 raft_consensus.cc:348] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.680994 25814 raft_consensus.cc:374] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:32.681121 25814 raft_consensus.cc:729] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95c2761f05346d6b751b3eb9604deb8, State: Initialized, Role: FOLLOWER
I20260810 18:52:32.680994 25817 raft_consensus.cc:348] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.681392 25817 raft_consensus.cc:374] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:52:32.681520 25817 raft_consensus.cc:729] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68972503abf440109bf8773513d2be46, State: Initialized, Role: FOLLOWER
I20260810 18:52:32.681713 25814 consensus_queue.cc:260] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.682075 25817 consensus_queue.cc:260] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.682977 25793 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:32.683637 25814 ts_tablet_manager.cc:1428] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 18:52:32.684624 25574 heartbeater.cc:503] Master 127.29.174.254:39349 was elected leader, sending a full tablet report...
I20260810 18:52:32.685031 25817 ts_tablet_manager.cc:1428] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260810 18:52:32.792773 25825 raft_consensus.cc:482] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:52:32.793077 25825 raft_consensus.cc:504] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.794876 25825 leader_election.cc:288] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503), f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481)
W20260810 18:52:32.800292 25794 tablet.cc:2369] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:32.801688 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.795515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:53771 (local address 127.29.174.193:35481)
0810 18:52:32.795711 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.795717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.796356 (+ 639us) server_negotiation.cc:407] Connection header received
0810 18:52:32.796472 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.796477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.796554 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.796647 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:32.797320 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.798149 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.799992 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.800831 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.800982 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.801058 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.801374 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.801472 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:52:32.801966 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.795194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:53709 (local address 127.29.174.195:37503)
0810 18:52:32.795429 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.795436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.796121 (+ 685us) server_negotiation.cc:407] Connection header received
0810 18:52:32.796313 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.796318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.796394 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.796482 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:52:32.797137 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.797899 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.799824 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.801099 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.801263 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.801318 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.801634 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.801758 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:52:32.801967 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.795348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35481 (local address 127.29.174.194:53771)
0810 18:52:32.796209 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.796223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.796355 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.796678 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.796681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.796702 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:32.797136 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.797148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.798321 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.798325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.799828 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.799842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.801028 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.801034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.801199 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.801233 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.801339 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.801468 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":3,"thread_start_us":539,"threads_started":1}
I20260810 18:52:32.802127 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.795055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:37503 (local address 127.29.174.194:53709)
0810 18:52:32.795990 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:32.796018 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:52:32.796173 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:52:32.796507 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:52:32.796511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:52:32.796535 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:52:32.796990 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.797003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.798059 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.798064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.799658 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:52:32.799672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.801323 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.801330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:52:32.801483 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.801513 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:52:32.801604 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:52:32.801715 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":18,"thread_start_us":577,"threads_started":1}
I20260810 18:52:32.803102 25703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879e7313e84a478c8171869e3cc63bea" candidate_uuid: "68972503abf440109bf8773513d2be46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95c2761f05346d6b751b3eb9604deb8" is_pre_election: true
I20260810 18:52:32.803102 25277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879e7313e84a478c8171869e3cc63bea" candidate_uuid: "68972503abf440109bf8773513d2be46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a8e5c75c484867be80f4cb36c02207" is_pre_election: true
I20260810 18:52:32.803640 25277 raft_consensus.cc:2454] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68972503abf440109bf8773513d2be46 in term 0.
I20260810 18:52:32.803653 25703 raft_consensus.cc:2454] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68972503abf440109bf8773513d2be46 in term 0.
I20260810 18:52:32.804273 25389 leader_election.cc:302] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68972503abf440109bf8773513d2be46, d95c2761f05346d6b751b3eb9604deb8; no voters:
I20260810 18:52:32.804680 25825 raft_consensus.cc:2781] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:52:32.804827 25825 raft_consensus.cc:482] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:52:32.804926 25825 raft_consensus.cc:3037] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:32.807272 25825 raft_consensus.cc:504] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.808041 25825 leader_election.cc:288] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [CANDIDATE]: Term 1 election: Requested vote from peers d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503), f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481)
I20260810 18:52:32.808496 25277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879e7313e84a478c8171869e3cc63bea" candidate_uuid: "68972503abf440109bf8773513d2be46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a8e5c75c484867be80f4cb36c02207"
I20260810 18:52:32.808486 25703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "879e7313e84a478c8171869e3cc63bea" candidate_uuid: "68972503abf440109bf8773513d2be46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d95c2761f05346d6b751b3eb9604deb8"
I20260810 18:52:32.808682 25277 raft_consensus.cc:3037] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:32.808688 25703 raft_consensus.cc:3037] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:52:32.811103 25277 raft_consensus.cc:2454] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68972503abf440109bf8773513d2be46 in term 1.
I20260810 18:52:32.811584 25390 leader_election.cc:302] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68972503abf440109bf8773513d2be46, f2a8e5c75c484867be80f4cb36c02207; no voters:
I20260810 18:52:32.811746 25703 raft_consensus.cc:2454] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68972503abf440109bf8773513d2be46 in term 1.
I20260810 18:52:32.811848 25825 raft_consensus.cc:2781] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:52:32.812330 25825 raft_consensus.cc:686] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 1 LEADER]: Becoming Leader. State: Replica: 68972503abf440109bf8773513d2be46, State: Running, Role: LEADER
I20260810 18:52:32.812783 25825 consensus_queue.cc:237] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } }
I20260810 18:52:32.817514 25040 catalog_manager.cc:5495] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68972503abf440109bf8773513d2be46 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "68972503abf440109bf8773513d2be46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68972503abf440109bf8773513d2be46" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 40411 } health_report { overall_health: HEALTHY } } }
W20260810 18:52:32.824774 25576 tablet.cc:2369] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:52:32.831741 25353 tablet.cc:2369] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:52:32.873893 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.869048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43366 (local address 127.29.174.194:40411)
0810 18:52:32.869221 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.869226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.869405 (+ 179us) server_negotiation.cc:407] Connection header received
0810 18:52:32.869573 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.869577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.869667 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.869797 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:52:32.870078 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.870816 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.872029 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.872440 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.872535 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.872977 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:52:32.873446 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:52:32.873548 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.873613 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.873675 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:52:32.881323 25277 raft_consensus.cc:1261] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Refusing update from remote peer 68972503abf440109bf8773513d2be46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:32.881327 25703 raft_consensus.cc:1261] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Refusing update from remote peer 68972503abf440109bf8773513d2be46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:52:32.882171 25829 consensus_queue.cc:1035] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d95c2761f05346d6b751b3eb9604deb8" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 37503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:32.882529 25825 consensus_queue.cc:1035] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a8e5c75c484867be80f4cb36c02207" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:52:32.910516 25044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:32.910789 25044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:52:32.914060 25044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Sending DeleteTablet for 3 replicas of tablet 879e7313e84a478c8171869e3cc63bea
I20260810 18:52:32.915108 25678 tablet_service.cc:1514] Processing DeleteTablet for tablet 879e7313e84a478c8171869e3cc63bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:32 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:39778
I20260810 18:52:32.915212 25244 tablet_service.cc:1514] Processing DeleteTablet for tablet 879e7313e84a478c8171869e3cc63bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:32 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:41156
I20260810 18:52:32.915510 25460 tablet_service.cc:1514] Processing DeleteTablet for tablet 879e7313e84a478c8171869e3cc63bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:52:32 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:43350
I20260810 18:52:32.916050 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 25114
I20260810 18:52:32.917207 25863 tablet_replica.cc:330] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: stopping tablet replica
I20260810 18:52:32.917400 25864 tablet_replica.cc:330] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: stopping tablet replica
I20260810 18:52:32.917658 25863 raft_consensus.cc:2229] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:32.917773 25864 raft_consensus.cc:2229] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:52:32.918193 25863 raft_consensus.cc:2258] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:32.918224 25864 raft_consensus.cc:2258] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:32.918972 25865 tablet_replica.cc:330] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: stopping tablet replica
I20260810 18:52:32.919390 25865 raft_consensus.cc:2229] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:52:32.920069 25865 raft_consensus.cc:2258] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:52:32.920925 25863 ts_tablet_manager.cc:1905] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:32.921948 25864 ts_tablet_manager.cc:1905] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:32.928894 25864 ts_tablet_manager.cc:1918] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:32.928890 25863 ts_tablet_manager.cc:1918] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:32.929078 25864 log.cc:1196] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/879e7313e84a478c8171869e3cc63bea
I20260810 18:52:32.929118 25865 ts_tablet_manager.cc:1905] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:52:32.929193 25863 log.cc:1196] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/879e7313e84a478c8171869e3cc63bea
I20260810 18:52:32.929674 25864 ts_tablet_manager.cc:1939] T 879e7313e84a478c8171869e3cc63bea P f2a8e5c75c484867be80f4cb36c02207: Deleting consensus metadata
I20260810 18:52:32.929713 25863 ts_tablet_manager.cc:1939] T 879e7313e84a478c8171869e3cc63bea P d95c2761f05346d6b751b3eb9604deb8: Deleting consensus metadata
I20260810 18:52:32.931486 25013 catalog_manager.cc:4841] TS d95c2761f05346d6b751b3eb9604deb8 (127.29.174.195:37503): tablet 879e7313e84a478c8171869e3cc63bea (table test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]) successfully deleted
I20260810 18:52:32.931474 25015 catalog_manager.cc:4841] TS f2a8e5c75c484867be80f4cb36c02207 (127.29.174.193:35481): tablet 879e7313e84a478c8171869e3cc63bea (table test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]) successfully deleted
I20260810 18:52:32.937258 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:32.917636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.29.174.193:35481)
0810 18:52:32.917815 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:32.917821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:32.917841 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:52:32.917907 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:32.917913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:32.918001 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:32.918095 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:32.919979 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.921132 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.923215 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:32.923919 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:32.924048 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:32.929436 (+ 5388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:32.929453 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:32.929470 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:32.929515 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:32.933730 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.934663 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.934673 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.934677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.934868 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:32.935338 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:32.935343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:32.935347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:32.935801 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:32.936100 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:32.936471 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 18:52:32.936704 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:52:32.937733 25865 ts_tablet_manager.cc:1918] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:52:32.937925 25865 log.cc:1196] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/879e7313e84a478c8171869e3cc63bea
I20260810 18:52:32.937978 25344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:32.938578 25865 ts_tablet_manager.cc:1939] T 879e7313e84a478c8171869e3cc63bea P 68972503abf440109bf8773513d2be46: Deleting consensus metadata
I20260810 18:52:32.940423 25016 catalog_manager.cc:4841] TS 68972503abf440109bf8773513d2be46 (127.29.174.194:40411): tablet 879e7313e84a478c8171869e3cc63bea (table test-table [id=572638e3c0f544d7ae6b9e7a02de8ee5]) successfully deleted
I20260810 18:52:33.082379 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 25114
I20260810 18:52:33.111935 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 25357
I20260810 18:52:33.126142 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:33.112736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.29.174.194:40411)
0810 18:52:33.112915 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:33.112920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:33.113018 (+ 98us) server_negotiation.cc:407] Connection header received
0810 18:52:33.113193 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:33.113199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:33.113266 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:33.113370 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:33.114726 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:33.115614 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:33.117046 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:33.117508 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:33.117617 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:33.120248 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:33.120259 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:33.120274 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:33.120309 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:33.123241 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:33.123994 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:33.124002 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:33.124005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:33.124176 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:33.124549 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:33.124553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:33.124555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:33.124963 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:33.125297 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:33.125600 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:52:33.125815 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:52:33.126905 25567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:33.238763 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 25357
I20260810 18:52:33.263063 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 25579
I20260810 18:52:33.276201 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:33.263802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.29.174.195:37503)
0810 18:52:33.264003 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:52:33.264009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:52:33.264030 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:52:33.264251 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:52:33.264257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:52:33.264324 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:52:33.264422 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:52:33.265908 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:33.266666 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:33.267955 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:52:33.268339 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:52:33.268450 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:52:33.271030 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:52:33.271042 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:52:33.271057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:52:33.271087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:52:33.273581 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:33.274214 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:33.274221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:33.274223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:33.274387 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:52:33.274748 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:52:33.274753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:52:33.274757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:52:33.275088 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:52:33.275368 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 18:52:33.275670 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 18:52:33.275887 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:52:33.276772 25786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:33.362164 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 25579
I20260810 18:52:33.387665 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 24987
I20260810 18:52:33.388746 25104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:52:33.498769 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 24987
I20260810 18:52:53.526244 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--webserver_interface=127.29.174.254
--webserver_port=45747
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:53.844666 25877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:53.844905 25877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:53.845028 25877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:53.851832 25877 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:52:53.851909 25877 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:52:53.851969 25877 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:52:53.852037 25877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:52:53.852104 25877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:53.852197 25877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:53.852257 25877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:52:53.852329 25877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:52:53.861424 25877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=45747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:53.862450 25877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:53.863618 25877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:53.874732 25886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:53.874778 25893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:53.874805 25889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:53.875169 25877 server_base.cc:1034] running on GCE node
I20260810 18:52:53.876014 25877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:53.877164 25877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:53.878382 25877 hybrid_clock.cc:648] HybridClock initialized: now 1786387973878318 us; error 32 us; skew 500 ppm
I20260810 18:52:53.878661 25877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:53 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387973, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:53.886006 25877 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:52:53.887387 25877 webserver.cc:460] Webserver started at http://127.29.174.254:45747/ using document root <none> and password file <none>
I20260810 18:52:53.887972 25877 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:53.888052 25877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:53.893020 25877 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:52:53.895632 25902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:53.896778 25877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:52:53.896904 25877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "462e0e7ccb8248f8a51cd7589cdf4554"
format_stamp: "Formatted at 2026-08-10 18:52:07 on dist-test-slave-6zbq"
server_key: "d937507dd69fe42d6cd3de10767a9793"
server_key_iv: "6abea1c51e9a6f28c355688275313b8f"
server_key_version: "encryptionkey@0"
I20260810 18:52:53.897737 25877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:53.910755 25877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:53.914026 25877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:53.914242 25877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:53.936185 25877 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39349
I20260810 18:52:53.936276 25987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39349 every 8 connection(s)
I20260810 18:52:53.937565 25877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:52:53.942903 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 25877
I20260810 18:52:53.943934 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:35481
--local_ip_for_outbound_sockets=127.29.174.193
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=40319
--webserver_interface=127.29.174.193
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:52:53.946259 25991 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:52:53.949581 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap starting.
I20260810 18:52:53.975633 25991 log.cc:824] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Log is configured to *not* fsync() on all Append() calls
I20260810 18:52:53.992446 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:52:53.992970 25991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Bootstrap complete.
I20260810 18:52:54.002099 25991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:54.003051 25991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Initialized, Role: FOLLOWER
I20260810 18:52:54.003945 25991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:54.004542 25991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:52:54.004746 25991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:52:54.004923 25991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:52:54.009172 25991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:54.009661 25991 leader_election.cc:302] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 462e0e7ccb8248f8a51cd7589cdf4554; no voters:
I20260810 18:52:54.010310 25991 leader_election.cc:288] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:52:54.010541 25995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:52:54.011499 25995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [term 3 LEADER]: Becoming Leader. State: Replica: 462e0e7ccb8248f8a51cd7589cdf4554, State: Running, Role: LEADER
I20260810 18:52:54.012173 25995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } }
I20260810 18:52:54.012336 25991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:52:54.014706 25998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 462e0e7ccb8248f8a51cd7589cdf4554. Latest consensus state: current_term: 3 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:54.015069 25998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:54.015623 25996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "462e0e7ccb8248f8a51cd7589cdf4554" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39349 } } }
I20260810 18:52:54.015913 25996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554 [sys.catalog]: This master's current role is: LEADER
I20260810 18:52:54.018577 26004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:52:54.027377 26004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:52:54.028806 26004 catalog_manager.cc:1259] Loaded cluster ID: df7d73688feb45378cc62205b99971d1
I20260810 18:52:54.028932 26004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:52:54.032858 26004 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:52:54.034624 26004 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Loaded TSK: 0
I20260810 18:52:54.034796 26004 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 462e0e7ccb8248f8a51cd7589cdf4554: Loaded TSK: 1
F20260810 18:52:54.035877 26004 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6ae14569ad google::LogMessage::Fail() at ??:0
@ 0x7f6ae145988a google::LogMessage::SendToLog() at ??:0
@ 0x7f6ae14564de google::LogMessage::Flush() at ??:0
@ 0x7f6ae1457a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260810 18:52:54.046056 26026 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f6ae51ba7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6ae51a973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6af6e851c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6af6e84783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6af6f102a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f6af6f100c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f6af6f5a5e9 std::function<>::operator()() at ??:0
@ 0x7f6af6e88cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6af6e86c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6af6f0e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6af6f0e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6ae40a2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6ae40a591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6ae40a5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6ae4080eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6ae735a6db start_thread at ??:0
@ 0x7f6ae02f061f clone at ??:0
W20260810 18:52:54.324558 25993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:54.324810 25993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:54.324928 25993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:54.331965 25993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:54.332057 25993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:54.332198 25993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:52:54.341423 25993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:35481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=40319
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:54.342630 25993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:54.343856 25993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:54.356038 26063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:54.356133 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:54.356290 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:54.357367 25993 server_base.cc:1034] running on GCE node
I20260810 18:52:54.357914 25993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:54.358803 25993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:54.359980 25993 hybrid_clock.cc:648] HybridClock initialized: now 1786387974359930 us; error 46 us; skew 500 ppm
I20260810 18:52:54.360249 25993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:54 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:54.363117 25993 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:52:54.364380 25993 webserver.cc:460] Webserver started at http://127.29.174.193:40319/ using document root <none> and password file <none>
I20260810 18:52:54.365002 25993 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:54.365114 25993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:54.370014 25993 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:52:54.372623 26077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.373812 25993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:52:54.373960 25993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "f2a8e5c75c484867be80f4cb36c02207"
format_stamp: "Formatted at 2026-08-10 18:52:08 on dist-test-slave-6zbq"
server_key: "107fb834a6b20502ab1ee141ff75f10a"
server_key_iv: "a48d2dd65cbf017717ef79b833cb3d0c"
server_key_version: "encryptionkey@0"
I20260810 18:52:54.374918 25993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:54.385977 25993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:54.389379 25993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:54.389605 25993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:54.390506 25993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:54.391690 25993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:54.391801 25993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.391902 25993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:54.391974 25993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.421602 25993 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35481
I20260810 18:52:54.421674 26287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35481 every 8 connection(s)
I20260810 18:52:54.422916 25993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:52:54.427172 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.425448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:41913)
0810 18:52:54.426134 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.426373 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":272,"threads_started":1}
W20260810 18:52:54.427821 26289 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:39349 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:39349: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
I20260810 18:52:54.428530 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.428233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:43065)
0810 18:52:54.428367 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.428431 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:52:54.429342 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.429019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:32803)
0810 18:52:54.429151 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.429225 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:52:54.429407 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 25993
W20260810 18:52:54.429571 26289 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:52:54.430109 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:40411
--local_ip_for_outbound_sockets=127.29.174.194
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=33717
--webserver_interface=127.29.174.194
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:54.747628 26292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:54.747865 26292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:54.747951 26292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:54.754905 26292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:54.754993 26292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:54.755125 26292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:52:54.764081 26292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:40411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=33717
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:54.765086 26292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:54.766202 26292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:54.777946 26303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:54.777997 26304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:54.778024 26307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:54.779330 26292 server_base.cc:1034] running on GCE node
I20260810 18:52:54.779897 26292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:54.780740 26292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:54.781929 26292 hybrid_clock.cc:648] HybridClock initialized: now 1786387974781889 us; error 31 us; skew 500 ppm
I20260810 18:52:54.782210 26292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:54 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387974, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:54.785246 26292 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:52:54.786579 26292 webserver.cc:460] Webserver started at http://127.29.174.194:33717/ using document root <none> and password file <none>
I20260810 18:52:54.787181 26292 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:54.787297 26292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:54.792167 26292 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 18:52:54.794624 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.795763 26292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:52:54.795910 26292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "68972503abf440109bf8773513d2be46"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "b31606ea670ca398d3d5a150363eade7"
server_key_iv: "0e093466b375c04c1eecce5cebfc85d1"
server_key_version: "encryptionkey@0"
I20260810 18:52:54.796805 26292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:54.818818 26292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:54.822327 26292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:54.822579 26292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:54.823464 26292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:54.824740 26292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:54.824833 26292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.824939 26292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:54.825008 26292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:54.854018 26292 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:40411
I20260810 18:52:54.854084 26510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:40411 every 8 connection(s)
I20260810 18:52:54.855274 26292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:52:54.859469 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.857810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:57071)
0810 18:52:54.858487 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.858711 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":248,"threads_started":1}
W20260810 18:52:54.860025 26513 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:39349 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:39349: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
I20260810 18:52:54.860699 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.860421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:39469)
0810 18:52:54.860532 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.860591 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 18:52:54.861394 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:54.861133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:47753)
0810 18:52:54.861231 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:54.861295 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 18:52:54.861594 26513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:52:54.865450 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 26292
I20260810 18:52:54.866088 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:37503
--local_ip_for_outbound_sockets=127.29.174.195
--tserver_master_addrs=127.29.174.254:39349
--webserver_port=40673
--webserver_interface=127.29.174.195
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:52:55.178232 26516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:52:55.178473 26516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:52:55.178573 26516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:52:55.185390 26516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:52:55.185499 26516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:52:55.185650 26516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:52:55.194571 26516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:37503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=40673
--tserver_master_addrs=127.29.174.254:39349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:52:55.195616 26516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:52:55.196785 26516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:52:55.208842 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:55.208802 26523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:52:55.208877 26526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:52:55.209784 26516 server_base.cc:1034] running on GCE node
I20260810 18:52:55.210462 26516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:52:55.211326 26516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:52:55.212500 26516 hybrid_clock.cc:648] HybridClock initialized: now 1786387975212457 us; error 42 us; skew 500 ppm
I20260810 18:52:55.212762 26516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:52:55 dist-test-slave-6zbq krb5kdc[24076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387975, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:52:55.215643 26516 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:52:55.216871 26516 webserver.cc:460] Webserver started at http://127.29.174.195:40673/ using document root <none> and password file <none>
I20260810 18:52:55.217427 26516 fs_manager.cc:359] Metadata directory not provided
I20260810 18:52:55.217522 26516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:52:55.222312 26516 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:52:55.224880 26539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:55.226055 26516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:52:55.226197 26516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "d95c2761f05346d6b751b3eb9604deb8"
format_stamp: "Formatted at 2026-08-10 18:52:09 on dist-test-slave-6zbq"
server_key: "bbe12c70f4b4a3e7fd40616f407e0dc8"
server_key_iv: "1f2319dd766339006a616a2ef8f30686"
server_key_version: "encryptionkey@0"
I20260810 18:52:55.227144 26516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:52:55.269606 26516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:52:55.273626 26516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:52:55.273874 26516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:52:55.274825 26516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:52:55.276083 26516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:52:55.276168 26516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:55.276273 26516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:52:55.276340 26516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:52:55.305594 26516 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:37503
I20260810 18:52:55.305680 26727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:37503 every 8 connection(s)
I20260810 18:52:55.306907 26516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:52:55.311618 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 26516
I20260810 18:52:55.311997 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.309655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:38603)
0810 18:52:55.310914 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:55.311170 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":532,"threads_started":1}
W20260810 18:52:55.312618 26728 heartbeater.cc:644] Failed to heartbeat to 127.29.174.254:39349 (0 consecutive failures): Network error: Failed to ping master at 127.29.174.254:39349: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
I20260810 18:52:55.313380 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.313077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:36475)
0810 18:52:55.313218 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:55.313282 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 18:52:55.314144 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.313840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:46165)
0810 18:52:55.313972 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:55.314043 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:52:55.314392 26728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:52:55.430789 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.430017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:57129)
0810 18:52:55.430487 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:55.430563 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":204,"threads_started":1}
I20260810 18:52:55.862977 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:55.862150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:50183)
0810 18:52:55.862661 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:55.862753 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":268,"threads_started":1}
I20260810 18:52:56.315546 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.314806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:37159)
0810 18:52:56.315246 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.315332 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":201,"threads_started":1}
I20260810 18:52:56.432188 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.431532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:56695)
0810 18:52:56.431964 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.432038 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":217,"threads_started":1}
I20260810 18:52:56.864420 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:56.863717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:40589)
0810 18:52:56.864181 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:56.864269 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":199,"threads_started":1}
I20260810 18:52:57.317536 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.316403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:36107)
0810 18:52:57.317295 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:57.317380 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":681,"threads_started":1}
I20260810 18:52:57.433552 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.432896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:58643)
0810 18:52:57.433311 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:57.433389 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":211,"threads_started":1}
I20260810 18:52:57.865868 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:57.865187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:57771)
0810 18:52:57.865620 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:57.865698 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":226,"threads_started":1}
I20260810 18:52:58.318981 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.318305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:45807)
0810 18:52:58.318747 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:58.318824 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":216,"threads_started":1}
I20260810 18:52:58.435055 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.434277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:40705)
0810 18:52:58.434725 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:58.434803 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260810 18:52:58.867312 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:58.866632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:48497)
0810 18:52:58.867050 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:58.867125 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":224,"threads_started":1}
I20260810 18:52:59.321218 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.319787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:49785)
0810 18:52:59.320934 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.321034 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":906,"threads_started":1}
I20260810 18:52:59.436663 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.435921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:54931)
0810 18:52:59.436415 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.436496 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260810 18:52:59.868788 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:52:59.868095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:53231)
0810 18:52:59.868541 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 18:52:59.868629 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260810 18:53:00.322656 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.321983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:54791)
0810 18:53:00.322425 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.322503 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":204,"threads_started":1}
I20260810 18:53:00.438053 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.437409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:58935)
0810 18:53:00.437811 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.437896 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":203,"threads_started":1}
I20260810 18:53:00.870160 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:00.869500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:45577)
0810 18:53:00.869932 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:00.870017 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":214,"threads_started":1}
I20260810 18:53:01.324038 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.323389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:41877)
0810 18:53:01.323806 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:01.323891 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":214,"threads_started":1}
I20260810 18:53:01.439518 26757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.438787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:52281)
0810 18:53:01.439228 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:01.439316 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":199,"threads_started":1}
I20260810 18:53:01.871776 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:01.870947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:49961)
0810 18:53:01.871412 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:01.871490 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":193,"threads_started":1}
I20260810 18:53:02.325650 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:02.324885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:54979)
0810 18:53:02.325370 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:02.325458 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":219,"threads_started":1}
I20260810 18:53:02.441044 26768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:02.440374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:53569)
0810 18:53:02.440803 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:02.440886 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":230,"threads_started":1}
I20260810 18:53:02.873260 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:02.872584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:34689)
0810 18:53:02.873015 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:02.873106 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":215,"threads_started":1}
I20260810 18:53:03.327170 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:03.326424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:50645)
0810 18:53:03.326878 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:03.326984 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":231,"threads_started":1}
I20260810 18:53:03.442521 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:03.441825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:34209)
0810 18:53:03.442250 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:03.442324 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":208,"threads_started":1}
I20260810 18:53:03.874723 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:03.874020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:39659)
0810 18:53:03.874480 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:03.874557 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":205,"threads_started":1}
I20260810 18:53:04.328720 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:04.328000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:36623)
0810 18:53:04.328458 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:04.328539 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":221,"threads_started":1}
I20260810 18:53:04.443976 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:04.443272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:55803)
0810 18:53:04.443735 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:04.443816 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":231,"threads_started":1}
I20260810 18:53:04.876183 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:04.875495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:50025)
0810 18:53:04.875931 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:04.876010 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260810 18:53:05.331276 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:05.329494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:55935)
0810 18:53:05.331004 (+ 1510us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:05.331114 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1396,"thread_start_us":1250,"threads_started":1}
I20260810 18:53:05.445533 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:05.444859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:55969)
0810 18:53:05.445287 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:05.445366 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":195,"threads_started":1}
I20260810 18:53:05.877636 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:05.876967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:56431)
0810 18:53:05.877371 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:05.877453 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":206,"threads_started":1}
I20260810 18:53:06.332700 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:06.332027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:34341)
0810 18:53:06.332434 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:06.332519 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":202,"threads_started":1}
I20260810 18:53:06.446919 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:06.446263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:40423)
0810 18:53:06.446698 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:06.446773 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":234,"threads_started":1}
I20260810 18:53:06.879088 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:06.878441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:47345)
0810 18:53:06.878843 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:06.878936 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":212,"threads_started":1}
I20260810 18:53:07.334069 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:07.333446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:58367)
0810 18:53:07.333851 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:07.333929 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":200,"threads_started":1}
I20260810 18:53:07.448410 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:07.447669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:34525)
0810 18:53:07.448119 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:07.448238 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":214,"threads_started":1}
I20260810 18:53:07.880524 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:07.879853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:58385)
0810 18:53:07.880257 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:07.880360 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":198,"threads_started":1}
I20260810 18:53:08.335665 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:08.334853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:42735)
0810 18:53:08.335319 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:08.335440 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":243,"threads_started":1}
I20260810 18:53:08.449806 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:08.449137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:50791)
0810 18:53:08.449552 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:08.449626 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":198,"threads_started":1}
I20260810 18:53:08.881923 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:08.881285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:35643)
0810 18:53:08.881666 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:08.881743 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":192,"threads_started":1}
I20260810 18:53:09.337193 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:09.336494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:42393)
0810 18:53:09.336912 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:09.337001 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":208,"threads_started":1}
I20260810 18:53:09.451467 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:09.450751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.193:47903)
0810 18:53:09.451196 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:09.451299 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":234,"threads_started":1}
I20260810 18:53:09.883600 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:09.882906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.194:37105)
0810 18:53:09.883328 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:09.883440 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":206,"threads_started":1}
I20260810 18:53:10.316227 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 25993
I20260810 18:53:10.339587 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:10.317448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.29.174.193:35481)
0810 18:53:10.318372 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:10.318393 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:10.318443 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:53:10.320096 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:10.320105 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:10.320532 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:10.320810 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:10.322152 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.323429 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.324684 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.325300 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.325427 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:10.333516 (+ 8089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:10.333525 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:10.333540 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:10.333580 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:10.336456 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.337034 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.337040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.337046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.337221 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.337535 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.337539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.337542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.338189 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:10.338478 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:10.338898 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 18:53:10.339258 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":543,"threads_started":1}
I20260810 18:53:10.339986 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:10.338233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39349 (local address 127.29.174.195:59151)
0810 18:53:10.339653 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:10.339800 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.254:39349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1305,"thread_start_us":1206,"threads_started":1}
I20260810 18:53:10.340937 26277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:10.423873 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 25993
I20260810 18:53:10.443689 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 26292
I20260810 18:53:10.460474 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:10.444541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35196 (local address 127.29.174.194:40411)
0810 18:53:10.445078 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:10.445092 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:10.445129 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:53:10.446791 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:10.446801 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:10.447212 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:10.447535 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:10.448958 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.450189 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.451469 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.452130 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.452310 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:10.454863 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:10.454872 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:10.454885 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:10.454923 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:10.457440 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.458059 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.458066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.458072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.458250 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.458624 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.458628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.458630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.459103 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:10.459408 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:10.459806 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 18:53:10.460182 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":238,"threads_started":1}
I20260810 18:53:10.461777 26506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:10.544076 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 26292
I20260810 18:53:10.567052 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 26516
I20260810 18:53:10.583089 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:10.568050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.29.174.195:37503)
0810 18:53:10.568617 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:10.568631 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:10.568673 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:53:10.570299 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:10.570308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:10.570748 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:10.571010 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:10.572274 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.573448 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.574684 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:10.575295 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:10.575419 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:10.577725 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:10.577733 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:10.577745 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:10.577781 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:10.580518 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.581058 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.581063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.581068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.581245 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:10.581523 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:10.581525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:10.581527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:10.581979 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:10.582210 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:10.582513 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:53:10.582810 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":269,"threads_started":1}
I20260810 18:53:10.584338 26718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:10.669359 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 26516
2026-08-10T18:53:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64481 ms)
[----------] 30 tests from SecurityITest (295311 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:10 dist-test-slave-6zbq krb5kdc[26824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:10 dist-test-slave-6zbq krb5kdc[26824](info): set up 2 sockets
Aug 10 18:53:10 dist-test-slave-6zbq krb5kdc[26824](info): commencing operation
krb5kdc: starting...
W20260810 18:53:11.901635 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.007s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:12 dist-test-slave-6zbq krb5kdc[26824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:53:12.151383 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35769
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:12.486976 26846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:12.487218 26846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:12.487340 26846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:12.494261 26846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:12.494381 26846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:12.494454 26846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:12.494530 26846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:12.494588 26846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:12.503881 26846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:12.505035 26846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:12.506232 26846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:12.517033 26863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.517078 26867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.517102 26857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:12.518635 26846 server_base.cc:1034] running on GCE node
I20260810 18:53:12.519313 26846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:12.520553 26846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:12.521780 26846 hybrid_clock.cc:648] HybridClock initialized: now 1786387992521732 us; error 28 us; skew 500 ppm
I20260810 18:53:12.522145 26846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:12 dist-test-slave-6zbq krb5kdc[26824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:12.525401 26846 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:53:12.526827 26846 webserver.cc:460] Webserver started at http://127.29.174.254:46731/ using document root <none> and password file <none>
I20260810 18:53:12.527432 26846 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:12.527544 26846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:12.527840 26846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:12.530709 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "669e6fe05c4c4910be9e4fc66cf88dd9"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-6zbq"
server_key: "16442b417b32b787186567a504ef0d6d"
server_key_iv: "246a2b89057b1369582020312290a8f4"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.531415 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "669e6fe05c4c4910be9e4fc66cf88dd9"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-6zbq"
server_key: "16442b417b32b787186567a504ef0d6d"
server_key_iv: "246a2b89057b1369582020312290a8f4"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.536239 26846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:12.539474 26886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.540619 26846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:53:12.540784 26846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "669e6fe05c4c4910be9e4fc66cf88dd9"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-6zbq"
server_key: "16442b417b32b787186567a504ef0d6d"
server_key_iv: "246a2b89057b1369582020312290a8f4"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.540953 26846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:12.562237 26846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:12.565671 26846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:12.565936 26846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:12.582525 26846 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35769
I20260810 18:53:12.582559 26969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35769 every 8 connection(s)
I20260810 18:53:12.583736 26846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:12.584486 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 26846
I20260810 18:53:12.584715 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:12.585412 30395 external_mini_cluster.cc:1428] Setting key 3c6e016b51189dad324f4d8f2ec52747
I20260810 18:53:12.587713 26970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:53:12 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:12.600644 26970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: Bootstrap starting.
I20260810 18:53:12.603754 26970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:12.605038 26970 log.cc:824] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:12.609366 26970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: No bootstrap required, opened a new log
I20260810 18:53:12.610033 26973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:12.587273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.29.174.254:35769)
0810 18:53:12.588109 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:12.588124 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:12.588174 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:53:12.590057 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:12.590078 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:12.590578 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:12.590915 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:12.592684 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.593952 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.595842 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.596477 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.596620 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:12.601347 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:12.601355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:12.601368 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:12.601402 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:12.604618 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.605453 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.605478 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.605486 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.605693 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.606105 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.606125 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.606129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.606888 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:12.607214 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:12.608350 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0810 18:53:12.608774 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":365,"threads_started":1}
I20260810 18:53:12.619777 26970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } }
I20260810 18:53:12.620268 26970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:12.620355 26970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669e6fe05c4c4910be9e4fc66cf88dd9, State: Initialized, Role: FOLLOWER
I20260810 18:53:12.620843 26970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } }
I20260810 18:53:12.621037 26970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:12.621165 26970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:12.621294 26970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:12.623850 26970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } }
I20260810 18:53:12.624300 26970 leader_election.cc:302] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 669e6fe05c4c4910be9e4fc66cf88dd9; no voters:
I20260810 18:53:12.624810 26970 leader_election.cc:288] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:12.624943 26975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:12.625562 26975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [term 1 LEADER]: Becoming Leader. State: Replica: 669e6fe05c4c4910be9e4fc66cf88dd9, State: Running, Role: LEADER
I20260810 18:53:12.626049 26975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } }
I20260810 18:53:12.626783 26970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:12.628762 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } } }
I20260810 18:53:12.629194 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:12.629861 26979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 669e6fe05c4c4910be9e4fc66cf88dd9. Latest consensus state: current_term: 1 leader_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669e6fe05c4c4910be9e4fc66cf88dd9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35769 } } }
I20260810 18:53:12.630230 26979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:12.631508 26988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:12.635398 26988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:12.643659 26988 catalog_manager.cc:1347] Generated new cluster ID: 80cf2b0ebb924a46827f11965c375d1d
I20260810 18:53:12.643764 26988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:12.652438 26988 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:12.653630 26988 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:12.664156 26988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: Generated new TSK 0
I20260810 18:53:12.664799 26988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:53:12.726552 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:13.055337 27011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:13.055573 27011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:13.055713 27011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:13.062646 27011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:13.062748 27011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:13.062898 27011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:13.071980 27011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:13.073077 27011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:13.074278 27011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:13.086503 27023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:13.086566 27024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:13.086469 27028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:13.091558 27011 server_base.cc:1034] running on GCE node
I20260810 18:53:13.092099 27011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:13.092927 27011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:13.094126 27011 hybrid_clock.cc:648] HybridClock initialized: now 1786387993094088 us; error 24 us; skew 500 ppm
I20260810 18:53:13.094444 27011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:13 dist-test-slave-6zbq krb5kdc[26824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:13.097327 27011 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:53:13.098744 27011 webserver.cc:460] Webserver started at http://127.29.174.193:35877/ using document root <none> and password file <none>
I20260810 18:53:13.099363 27011 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:13.099475 27011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:13.099768 27011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:13.102517 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "ede2757d6619426db9d66cf8ba292d08"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "df69c6e19b4f5a4090f8546c418fb299"
server_key_iv: "6227b6539dd3b34717df25e982a2ec7a"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.103257 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "ede2757d6619426db9d66cf8ba292d08"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "df69c6e19b4f5a4090f8546c418fb299"
server_key_iv: "6227b6539dd3b34717df25e982a2ec7a"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.107908 27011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:13.111212 27037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.112356 27011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:53:13.112502 27011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "ede2757d6619426db9d66cf8ba292d08"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "df69c6e19b4f5a4090f8546c418fb299"
server_key_iv: "6227b6539dd3b34717df25e982a2ec7a"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.112694 27011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:13.129196 27011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:13.132954 27011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:13.133186 27011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:13.133971 27011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:13.135247 27011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:13.135316 27011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.135408 27011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:13.135452 27011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.163769 27011 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:34371
I20260810 18:53:13.163873 27221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:34371 every 8 connection(s)
I20260810 18:53:13.164942 27011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:13.172569 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 27011
I20260810 18:53:13.172776 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:13.173451 30395 external_mini_cluster.cc:1428] Setting key f543eccbb165706abad27e466ba598b3
Aug 10 18:53:13 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:13.183697 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.167338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:49171 (local address 127.29.174.254:35769)
0810 18:53:13.167756 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.167761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.168698 (+ 937us) server_negotiation.cc:407] Connection header received
0810 18:53:13.170403 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.170407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.170471 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.170564 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.171870 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.172870 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.174552 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.175149 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.175265 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.178506 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.178513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.178516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.178548 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.180841 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.181513 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.181520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.181524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.181645 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.182044 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.182049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.182053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.182306 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.182462 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.183099 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.183329 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":201,"threads_started":1}
I20260810 18:53:13.184507 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.167869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35769 (local address 127.29.174.193:49171)
0810 18:53:13.168529 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:13.168565 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:13.170145 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:13.170743 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:13.170750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:13.171233 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:13.171742 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.171758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.173026 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.173030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.174422 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.174435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.175374 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.175382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.175601 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.176221 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:13.176246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:13.178284 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:13.180968 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.180977 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.180991 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.181402 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.181755 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.181760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.181764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.181940 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.182443 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:13.182450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:13.182752 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 18:53:13.182983 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 18:53:13.183456 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":261,"threads_started":1}
I20260810 18:53:13.185911 27222 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35769
I20260810 18:53:13.186168 27222 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:13.186862 27222 heartbeater.cc:511] Master 127.29.174.254:35769 requested a full tablet report, sending...
I20260810 18:53:13.188730 26916 ts_manager.cc:194] Registered new tserver with Master: ede2757d6619426db9d66cf8ba292d08 (127.29.174.193:34371)
I20260810 18:53:13.190114 26916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:49171
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:53:13.233752 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:13.552815 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:13.553042 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:13.553154 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:13.560035 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:13.560132 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:13.560241 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:13.569204 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:13.570180 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:13.571324 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:13.583235 27246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:13.583427 27251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:13.583822 27237 server_base.cc:1034] running on GCE node
W20260810 18:53:13.588811 27245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:13.589411 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:13.590255 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:13.591444 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786387993591388 us; error 51 us; skew 500 ppm
I20260810 18:53:13.591720 27237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:13 dist-test-slave-6zbq krb5kdc[26824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:13.594594 27237 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:53:13.595875 27237 webserver.cc:460] Webserver started at http://127.29.174.194:34675/ using document root <none> and password file <none>
I20260810 18:53:13.596454 27237 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:13.596555 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:13.596834 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:13.599566 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "8f8e6418c38140cba9989b590e1828be"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "ec179d570155b8d55c5e4e886adb6c57"
server_key_iv: "5b1cd1743d54207368bfb34b8cee3a09"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.600255 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "8f8e6418c38140cba9989b590e1828be"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "ec179d570155b8d55c5e4e886adb6c57"
server_key_iv: "5b1cd1743d54207368bfb34b8cee3a09"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.604854 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:13.607945 27260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.609071 27237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:13.609212 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "8f8e6418c38140cba9989b590e1828be"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-6zbq"
server_key: "ec179d570155b8d55c5e4e886adb6c57"
server_key_iv: "5b1cd1743d54207368bfb34b8cee3a09"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.609385 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:13.642503 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:13.646049 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:13.646301 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:13.647166 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:13.648353 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:13.648422 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.648540 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:13.648586 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.677932 27237 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:42733
I20260810 18:53:13.677992 27453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:42733 every 8 connection(s)
I20260810 18:53:13.679112 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:13.679560 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 27237
I20260810 18:53:13.679740 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:13.680330 30395 external_mini_cluster.cc:1428] Setting key c63db77d2b7f92ff767464a240f1467d
Aug 10 18:53:13 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:13.699147 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.681582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:52769 (local address 127.29.174.254:35769)
0810 18:53:13.681783 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.681789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.683036 (+ 1247us) server_negotiation.cc:407] Connection header received
0810 18:53:13.685334 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.685340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.685420 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.685514 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.687252 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.687943 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.689845 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.690246 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.690329 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.693960 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.693966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.693969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.694000 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.696294 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.697063 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.697067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.697069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.697186 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.697535 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.697538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.697540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.697783 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.697912 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.698575 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.698822 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:13.699921 27460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.682109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35769 (local address 127.29.174.194:52769)
0810 18:53:13.682880 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:13.682924 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:13.685072 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:13.685765 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:13.685774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:13.686424 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:13.687106 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.687128 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.688087 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.688092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.689733 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.689747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.690470 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.690477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.690755 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.691346 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:13.691376 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:13.693749 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:13.696422 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.696430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.696444 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.696959 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.697279 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.697282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.697285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.697453 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.697911 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:13.697919 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:13.698191 (+ 272us) client_negotiation.cc:769] Sending connection context
0810 18:53:13.698453 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 18:53:13.698895 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":300,"threads_started":1}
I20260810 18:53:13.701184 27454 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35769
I20260810 18:53:13.701434 27454 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:13.702064 27454 heartbeater.cc:511] Master 127.29.174.254:35769 requested a full tablet report, sending...
I20260810 18:53:13.703397 26916 ts_manager.cc:194] Registered new tserver with Master: 8f8e6418c38140cba9989b590e1828be (127.29.174.194:42733)
I20260810 18:53:13.704139 26916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:52769
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:53:13.737167 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:14.069828 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:14.070169 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:14.070333 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:14.077527 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:14.077644 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:14.077827 27467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:14.088129 27467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:35769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:14.089426 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:14.090739 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:14.103171 27483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:14.103212 27475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:14.103235 27477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:14.103861 27467 server_base.cc:1034] running on GCE node
I20260810 18:53:14.104406 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:14.105284 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:14.106431 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786387994106379 us; error 39 us; skew 500 ppm
I20260810 18:53:14.106714 27467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:14 dist-test-slave-6zbq krb5kdc[26824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387994, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:14.109711 27467 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:53:14.111047 27467 webserver.cc:460] Webserver started at http://127.29.174.195:33089/ using document root <none> and password file <none>
I20260810 18:53:14.111680 27467 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:14.111781 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:14.112090 27467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:14.115015 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "1860c5b1cb584277a040141062a8a730"
format_stamp: "Formatted at 2026-08-10 18:53:14 on dist-test-slave-6zbq"
server_key: "afa4c2fd643120175ca70fb7f6489631"
server_key_iv: "92b0b36460c4bf5342b976fb812f12b1"
server_key_version: "encryptionkey@0"
I20260810 18:53:14.115731 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "1860c5b1cb584277a040141062a8a730"
format_stamp: "Formatted at 2026-08-10 18:53:14 on dist-test-slave-6zbq"
server_key: "afa4c2fd643120175ca70fb7f6489631"
server_key_iv: "92b0b36460c4bf5342b976fb812f12b1"
server_key_version: "encryptionkey@0"
I20260810 18:53:14.120740 27467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:53:14.123939 27499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:14.125133 27467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:14.125264 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "1860c5b1cb584277a040141062a8a730"
format_stamp: "Formatted at 2026-08-10 18:53:14 on dist-test-slave-6zbq"
server_key: "afa4c2fd643120175ca70fb7f6489631"
server_key_iv: "92b0b36460c4bf5342b976fb812f12b1"
server_key_version: "encryptionkey@0"
I20260810 18:53:14.125419 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:14.137439 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:14.141498 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:14.141739 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:14.142611 27467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:14.143818 27467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:14.143889 27467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:14.143990 27467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:14.144088 27467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:14.179018 27467 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:39011
I20260810 18:53:14.179052 27685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:39011 every 8 connection(s)
I20260810 18:53:14.180222 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:14.180692 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 27467
I20260810 18:53:14.180872 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:14.181458 30395 external_mini_cluster.cc:1428] Setting key 858ee8d74e1b0a3d768d259ddc62bc1b
I20260810 18:53:14.192782 27222 heartbeater.cc:503] Master 127.29.174.254:35769 was elected leader, sending a full tablet report...
Aug 10 18:53:14 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387994, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:14.201015 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.182857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56831 (local address 127.29.174.254:35769)
0810 18:53:14.183035 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.183041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.184855 (+ 1814us) server_negotiation.cc:407] Connection header received
0810 18:53:14.187183 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.187189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.187281 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.187367 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:14.188771 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.189531 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.191172 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.191595 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.191710 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.195079 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:14.195084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:14.195087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:14.195117 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:14.198132 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.198838 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.198842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.198844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.198969 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.199434 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.199440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.199442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.199738 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:14.199863 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.200598 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.200773 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:14.202009 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.183415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:35769 (local address 127.29.174.195:56831)
0810 18:53:14.184642 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.184702 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.186867 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.187557 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.187565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.188061 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:14.188618 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.188634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.189696 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.189700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.191012 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.191029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.191806 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.191811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.192005 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.192638 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:14.192660 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:14.194890 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:14.198263 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:14.198270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:14.198281 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:14.198738 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:14.199073 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:14.199076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:14.199078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:14.199283 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:14.199903 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:14.199909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:14.200216 (+ 307us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.200434 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.200931 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":315,"threads_started":1}
I20260810 18:53:14.203365 27687 heartbeater.cc:347] Connected to a master server at 127.29.174.254:35769
I20260810 18:53:14.203650 27687 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:14.204311 27687 heartbeater.cc:511] Master 127.29.174.254:35769 requested a full tablet report, sending...
I20260810 18:53:14.205919 26916 ts_manager.cc:194] Registered new tserver with Master: 1860c5b1cb584277a040141062a8a730 (127.29.174.195:39011)
I20260810 18:53:14.206852 26916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:56831
I20260810 18:53:14.209873 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:14.228242 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.214385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46344 (local address 127.29.174.254:35769)
0810 18:53:14.214536 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.214541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.214987 (+ 446us) server_negotiation.cc:407] Connection header received
0810 18:53:14.215117 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.215120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.215173 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.215262 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:14.216598 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.217356 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.218811 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.220249 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.220367 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.222981 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:14.222986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:14.222989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:14.223023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:14.225774 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.226506 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.226510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.226512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.226649 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.226939 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.226942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.226944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.227296 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:14.227487 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.227754 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.227917 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:14.232573 26916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:14.239382 26916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:14.263029 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.252010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.29.174.193:34371)
0810 18:53:14.253081 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.253092 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.254180 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 18:53:14.254296 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.254306 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.254602 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.255026 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:14.256005 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.257560 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.260399 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.261869 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.262095 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.262284 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.262533 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.262698 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":814,"threads_started":1}
I20260810 18:53:14.263126 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.251688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34371 (local address 127.0.0.1:50200)
0810 18:53:14.253695 (+ 2007us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.253750 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.253947 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.255355 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.255364 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.255398 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:14.255812 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.255824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.257744 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.257750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.260229 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.260244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.262105 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.262111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.262290 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.262362 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.262491 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.262686 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1729,"mutex_wait_us":3,"thread_start_us":4390,"threads_started":1}
I20260810 18:53:14.265079 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.256717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.29.174.195:39011)
0810 18:53:14.257327 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.257334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.257978 (+ 644us) server_negotiation.cc:407] Connection header received
0810 18:53:14.258241 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.258248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.258453 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.258629 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:14.259546 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.260810 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.262806 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.263938 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.264060 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.264165 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.264601 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.264779 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":289,"threads_started":1}
I20260810 18:53:14.265519 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.252665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.29.174.194:42733)
0810 18:53:14.256964 (+ 4299us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.256972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.256999 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:53:14.257092 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.257098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.257314 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.257835 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:14.258433 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.259787 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.261997 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.264540 (+ 2543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.264708 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.264882 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.265070 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.265226 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4148,"thread_start_us":4017,"threads_started":1}
I20260810 18:53:14.265717 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.251896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:42733 (local address 127.0.0.1:41706)
0810 18:53:14.256338 (+ 4442us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.256357 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.256462 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.257528 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.257534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.257561 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:14.258192 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.258218 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.259919 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.259924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.261834 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.261849 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.264730 (+ 2881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.264737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.264906 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.264938 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.265019 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.265141 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4316,"mutex_wait_us":24,"thread_start_us":2150,"threads_started":1}
I20260810 18:53:14.265946 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.257288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:39011 (local address 127.0.0.1:45812)
0810 18:53:14.257867 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.257884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.258076 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.258912 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.258916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.258939 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:14.259328 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.259340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.260937 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.260943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.262599 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.262614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.264303 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.264307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.264419 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.264439 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.265400 (+ 961us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.265536 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":248,"threads_started":1}
I20260810 18:53:14.267554 27112 tablet_service.cc:1467] Processing CreateTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 (DEFAULT_TABLE table=test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:14.269001 27112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762d0bfc131440a8ac4236fd84f2623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:14.269179 27349 tablet_service.cc:1467] Processing CreateTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 (DEFAULT_TABLE table=test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:14.269179 27576 tablet_service.cc:1467] Processing CreateTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 (DEFAULT_TABLE table=test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:14.270555 27349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762d0bfc131440a8ac4236fd84f2623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:14.270552 27576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762d0bfc131440a8ac4236fd84f2623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:14.279917 27716 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Bootstrap starting.
I20260810 18:53:14.283813 27716 tablet_bootstrap.cc:654] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:14.284557 27717 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Bootstrap starting.
I20260810 18:53:14.285162 27716 log.cc:824] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:14.286661 27719 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Bootstrap starting.
I20260810 18:53:14.288161 27716 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: No bootstrap required, opened a new log
I20260810 18:53:14.288298 27717 tablet_bootstrap.cc:654] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:14.288420 27716 ts_tablet_manager.cc:1397] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:53:14.289588 27717 log.cc:824] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:14.290235 27719 tablet_bootstrap.cc:654] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:14.291355 27719 log.cc:824] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:14.293035 27717 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: No bootstrap required, opened a new log
I20260810 18:53:14.293327 27717 ts_tablet_manager.cc:1397] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 18:53:14.294334 27719 tablet_bootstrap.cc:492] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: No bootstrap required, opened a new log
I20260810 18:53:14.294608 27719 ts_tablet_manager.cc:1397] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260810 18:53:14.296417 27716 raft_consensus.cc:348] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.296761 27716 raft_consensus.cc:374] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:14.296876 27716 raft_consensus.cc:729] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede2757d6619426db9d66cf8ba292d08, State: Initialized, Role: FOLLOWER
I20260810 18:53:14.297590 27716 consensus_queue.cc:260] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.299391 27716 ts_tablet_manager.cc:1428] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260810 18:53:14.303740 27717 raft_consensus.cc:348] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.304157 27717 raft_consensus.cc:374] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:14.304265 27717 raft_consensus.cc:729] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8e6418c38140cba9989b590e1828be, State: Initialized, Role: FOLLOWER
I20260810 18:53:14.304837 27717 consensus_queue.cc:260] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.305258 27719 raft_consensus.cc:348] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.305662 27719 raft_consensus.cc:374] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:14.305763 27719 raft_consensus.cc:729] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1860c5b1cb584277a040141062a8a730, State: Initialized, Role: FOLLOWER
I20260810 18:53:14.306388 27719 consensus_queue.cc:260] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.306672 27717 ts_tablet_manager.cc:1428] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260810 18:53:14.307497 27687 heartbeater.cc:503] Master 127.29.174.254:35769 was elected leader, sending a full tablet report...
I20260810 18:53:14.307988 27719 ts_tablet_manager.cc:1428] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260810 18:53:14.308290 27454 heartbeater.cc:503] Master 127.29.174.254:35769 was elected leader, sending a full tablet report...
I20260810 18:53:14.373370 27730 raft_consensus.cc:482] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:14.373657 27730 raft_consensus.cc:504] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.375612 27730 leader_election.cc:288] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ede2757d6619426db9d66cf8ba292d08 (127.29.174.193:34371), 1860c5b1cb584277a040141062a8a730 (127.29.174.195:39011)
I20260810 18:53:14.382236 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.376085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:57179 (local address 127.29.174.193:34371)
0810 18:53:14.376286 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.376291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.376806 (+ 515us) server_negotiation.cc:407] Connection header received
0810 18:53:14.377072 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.377078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.377171 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.377238 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:14.377878 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.378639 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.380584 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.381365 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.381509 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.381581 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.381870 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.381991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:14.382361 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.376918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:44831 (local address 127.29.174.195:39011)
0810 18:53:14.377115 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.377121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.377466 (+ 345us) server_negotiation.cc:407] Connection header received
0810 18:53:14.377645 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.377651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.377722 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.377796 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:14.378294 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.379045 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.380736 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.381490 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.381628 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.381708 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.381985 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.382105 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:53:14.382395 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.375972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34371 (local address 127.29.174.194:57179)
0810 18:53:14.376698 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.376723 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.376868 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.377421 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.377424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.377446 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:14.377737 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.377744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.378769 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.378773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.380438 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.380450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.381566 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.381572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.381709 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.381735 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.381820 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.381939 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":42,"thread_start_us":388,"threads_started":1}
I20260810 18:53:14.382509 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.376775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:39011 (local address 127.29.174.194:44831)
0810 18:53:14.377365 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:14.377378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:14.377508 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:14.377832 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:14.377836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:14.377859 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:14.378141 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.378149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.379191 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.379194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.380582 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:14.380594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.381689 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.381695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:14.381840 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.381864 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:14.381940 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:53:14.382059 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":1,"thread_start_us":285,"threads_started":1}
I20260810 18:53:14.383369 27603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762d0bfc131440a8ac4236fd84f2623" candidate_uuid: "8f8e6418c38140cba9989b590e1828be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1860c5b1cb584277a040141062a8a730" is_pre_election: true
I20260810 18:53:14.383391 27145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762d0bfc131440a8ac4236fd84f2623" candidate_uuid: "8f8e6418c38140cba9989b590e1828be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede2757d6619426db9d66cf8ba292d08" is_pre_election: true
I20260810 18:53:14.383824 27603 raft_consensus.cc:2454] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f8e6418c38140cba9989b590e1828be in term 0.
I20260810 18:53:14.383845 27145 raft_consensus.cc:2454] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f8e6418c38140cba9989b590e1828be in term 0.
I20260810 18:53:14.384409 27268 leader_election.cc:302] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8e6418c38140cba9989b590e1828be, ede2757d6619426db9d66cf8ba292d08; no voters:
I20260810 18:53:14.384756 27730 raft_consensus.cc:2781] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:14.384886 27730 raft_consensus.cc:482] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:14.384982 27730 raft_consensus.cc:3037] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:14.387624 27730 raft_consensus.cc:504] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.388170 27730 leader_election.cc:288] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [CANDIDATE]: Term 1 election: Requested vote from peers ede2757d6619426db9d66cf8ba292d08 (127.29.174.193:34371), 1860c5b1cb584277a040141062a8a730 (127.29.174.195:39011)
I20260810 18:53:14.388639 27145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762d0bfc131440a8ac4236fd84f2623" candidate_uuid: "8f8e6418c38140cba9989b590e1828be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede2757d6619426db9d66cf8ba292d08"
I20260810 18:53:14.388695 27603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762d0bfc131440a8ac4236fd84f2623" candidate_uuid: "8f8e6418c38140cba9989b590e1828be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1860c5b1cb584277a040141062a8a730"
I20260810 18:53:14.388825 27145 raft_consensus.cc:3037] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:14.388873 27603 raft_consensus.cc:3037] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:14.391000 27145 raft_consensus.cc:2454] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f8e6418c38140cba9989b590e1828be in term 1.
I20260810 18:53:14.391052 27603 raft_consensus.cc:2454] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f8e6418c38140cba9989b590e1828be in term 1.
I20260810 18:53:14.391348 27268 leader_election.cc:302] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8e6418c38140cba9989b590e1828be, ede2757d6619426db9d66cf8ba292d08; no voters:
I20260810 18:53:14.391628 27730 raft_consensus.cc:2781] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:14.392046 27730 raft_consensus.cc:686] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 1 LEADER]: Becoming Leader. State: Replica: 8f8e6418c38140cba9989b590e1828be, State: Running, Role: LEADER
I20260810 18:53:14.392484 27730 consensus_queue.cc:237] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } }
I20260810 18:53:14.396443 26915 catalog_manager.cc:5495] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f8e6418c38140cba9989b590e1828be (127.29.174.194). New cstate: current_term: 1 leader_uuid: "8f8e6418c38140cba9989b590e1828be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f8e6418c38140cba9989b590e1828be" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:14.416509 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.411274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.29.174.194:42733)
0810 18:53:14.411484 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.411490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.411505 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:14.411584 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.411588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.411682 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.411809 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:14.412225 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.412991 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.414440 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.414840 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.414951 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.415423 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:14.415912 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:14.416056 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.416170 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.416250 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:53:14.418424 27228 tablet.cc:2369] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:14.423942 27145 raft_consensus.cc:1261] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 1 FOLLOWER]: Refusing update from remote peer 8f8e6418c38140cba9989b590e1828be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:14.423938 27603 raft_consensus.cc:1261] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 1 FOLLOWER]: Refusing update from remote peer 8f8e6418c38140cba9989b590e1828be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:14.424854 27736 consensus_queue.cc:1035] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede2757d6619426db9d66cf8ba292d08" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:14.425348 27730 consensus_queue.cc:1035] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [LEADER]: Connected to new peer: Peer: permanent_uuid: "1860c5b1cb584277a040141062a8a730" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:53:14.431054 27458 tablet.cc:2369] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:14.432219 27689 tablet.cc:2369] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:14.439821 27742 mvcc.cc:205] Tried to move back new op lower bound from 7317045225151451136 to 7317045225035333632. Current Snapshot: MvccSnapshot[applied={T|T < 7317045225151451136}]
I20260810 18:53:14.454888 26915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:14.455138 26915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:14.458290 26915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 669e6fe05c4c4910be9e4fc66cf88dd9: Sending DeleteTablet for 3 replicas of tablet 5762d0bfc131440a8ac4236fd84f2623
I20260810 18:53:14.459372 27112 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:14 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:50200
I20260810 18:53:14.459440 27349 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:14 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:41706
I20260810 18:53:14.460094 27576 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762d0bfc131440a8ac4236fd84f2623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:14 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:45812
I20260810 18:53:14.462260 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 27011
I20260810 18:53:14.475813 27764 tablet_replica.cc:330] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: stopping tablet replica
I20260810 18:53:14.476349 27764 raft_consensus.cc:2229] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:14.476642 27767 tablet_replica.cc:330] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: stopping tablet replica
I20260810 18:53:14.476908 27764 raft_consensus.cc:2258] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:14.477039 27767 raft_consensus.cc:2229] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:14.477473 27767 raft_consensus.cc:2258] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:14.478266 27765 tablet_replica.cc:330] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: stopping tablet replica
I20260810 18:53:14.478719 27765 raft_consensus.cc:2229] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:14.479468 27765 raft_consensus.cc:2258] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:14.479625 27764 ts_tablet_manager.cc:1905] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:14.479658 27767 ts_tablet_manager.cc:1905] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:14.484090 27765 ts_tablet_manager.cc:1905] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:14.487062 27764 ts_tablet_manager.cc:1918] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:14.487310 27764 log.cc:1196] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/5762d0bfc131440a8ac4236fd84f2623
I20260810 18:53:14.487905 27764 ts_tablet_manager.cc:1939] T 5762d0bfc131440a8ac4236fd84f2623 P ede2757d6619426db9d66cf8ba292d08: Deleting consensus metadata
I20260810 18:53:14.488971 27767 ts_tablet_manager.cc:1918] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:14.489197 27767 log.cc:1196] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/5762d0bfc131440a8ac4236fd84f2623
I20260810 18:53:14.489876 26894 catalog_manager.cc:4841] TS ede2757d6619426db9d66cf8ba292d08 (127.29.174.193:34371): tablet 5762d0bfc131440a8ac4236fd84f2623 (table test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]) successfully deleted
I20260810 18:53:14.490535 27767 ts_tablet_manager.cc:1939] T 5762d0bfc131440a8ac4236fd84f2623 P 1860c5b1cb584277a040141062a8a730: Deleting consensus metadata
I20260810 18:53:14.492187 26894 catalog_manager.cc:4841] TS 1860c5b1cb584277a040141062a8a730 (127.29.174.195:39011): tablet 5762d0bfc131440a8ac4236fd84f2623 (table test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]) successfully deleted
I20260810 18:53:14.492231 27765 ts_tablet_manager.cc:1918] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:14.492385 27765 log.cc:1196] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/5762d0bfc131440a8ac4236fd84f2623
Aug 10 18:53:14 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:53:14.492969 27765 ts_tablet_manager.cc:1939] T 5762d0bfc131440a8ac4236fd84f2623 P 8f8e6418c38140cba9989b590e1828be: Deleting consensus metadata
I20260810 18:53:14.494563 26892 catalog_manager.cc:4841] TS 8f8e6418c38140cba9989b590e1828be (127.29.174.194:42733): tablet 5762d0bfc131440a8ac4236fd84f2623 (table test-table [id=b1cb67c7f3f44ca685d89d2a5ae2a3c5]) successfully deleted
I20260810 18:53:14.500541 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.463898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.29.174.193:34371)
0810 18:53:14.464073 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.464079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.476029 (+ 11950us) server_negotiation.cc:407] Connection header received
0810 18:53:14.476212 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.476218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.476312 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.476420 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:14.480672 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.481843 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.483698 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.484331 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.484427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.494043 (+ 9616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:14.494059 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:14.494078 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:14.494121 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:14.497772 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.498486 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.498496 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.498499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.498656 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.499012 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.499017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.499020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.499459 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:14.499788 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.499951 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.500169 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:14.501199 27214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:14.590864 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 27011
I20260810 18:53:14.619211 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 27237
Aug 10 18:53:14 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:53:14.636233 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.619974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.29.174.194:42733)
0810 18:53:14.620168 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.620174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.620205 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:53:14.620361 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.620367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.620465 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.620577 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:14.621775 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.623069 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.624676 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.625368 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.625501 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.630063 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:14.630078 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:14.630095 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:14.630143 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:14.633397 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.634233 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.634240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.634243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.634439 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.634852 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.634855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.634857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.635196 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:14.635455 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.635781 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.635941 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:14.637101 27448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:14.749596 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 27237
I20260810 18:53:14.771946 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 27467
Aug 10 18:53:14 dist-test-slave-6zbq krb5kdc[26824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:53:14.785580 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:14.772657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.29.174.195:39011)
0810 18:53:14.772846 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:14.772851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:14.772870 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:14.773002 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:14.773006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:14.773078 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:14.773160 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:14.774355 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.775087 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.776313 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:14.776693 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:14.776781 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:14.780802 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:14.780813 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:14.780828 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:14.780860 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:14.783410 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.783987 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.783993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.783995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.784106 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:14.784351 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:14.784354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:14.784355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:14.784672 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:14.784878 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:14.785081 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 18:53:14.785254 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:53:14.786222 27679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:14.873147 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 27467
I20260810 18:53:14.894183 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 26846
I20260810 18:53:14.895300 26963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:15.003890 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 26846
2026-08-10T18:53:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4340 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T18:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:15Z Disabled control of system clock
I20260810 18:53:15.091171 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:34165
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:34165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:15.415688 27782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.415935 27782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.416049 27782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.423033 27782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:15.423123 27782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.423168 27782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.423254 27782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:15.423305 27782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:15.432010 27782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:34165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:34165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:15.433109 27782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.434752 27782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.445430 27790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.445475 27793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.445509 27791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.446485 27782 server_base.cc:1034] running on GCE node
I20260810 18:53:15.447152 27782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.448364 27782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.449553 27782 hybrid_clock.cc:648] HybridClock initialized: now 1786387995449513 us; error 35 us; skew 500 ppm
I20260810 18:53:15.449869 27782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:15.455161 27782 webserver.cc:460] Webserver started at http://127.29.174.254:46785/ using document root <none> and password file <none>
I20260810 18:53:15.455783 27782 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.455878 27782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.456175 27782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.458961 27782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "49d03f0c0394435e94412f472c70b982"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "1d1fc0365f0dd1473f976b592de9c7a0"
server_key_iv: "08c6fdfd843f8e74300633e6b078c176"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.459721 27782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "49d03f0c0394435e94412f472c70b982"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "1d1fc0365f0dd1473f976b592de9c7a0"
server_key_iv: "08c6fdfd843f8e74300633e6b078c176"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.464638 27782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:53:15.467844 27806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.468973 27782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:15.469151 27782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "49d03f0c0394435e94412f472c70b982"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "1d1fc0365f0dd1473f976b592de9c7a0"
server_key_iv: "08c6fdfd843f8e74300633e6b078c176"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.469339 27782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.481050 27782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.481756 27782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.482005 27782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.499058 27782 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:34165
I20260810 18:53:15.499110 27889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:34165 every 8 connection(s)
I20260810 18:53:15.500211 27782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:15.502218 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 27782
I20260810 18:53:15.502386 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:15.502826 30395 external_mini_cluster.cc:1428] Setting key 3735ea1c7527fb6d15bd417307c3ed8a
I20260810 18:53:15.503753 27891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:15.516801 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.504492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.29.174.254:34165)
0810 18:53:15.505527 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.505541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.505588 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:53:15.507149 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.507174 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.507717 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.508117 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.508603 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.510565 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.512183 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.513134 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.513315 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.514266 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.514277 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.514293 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:15.514975 (+ 682us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.515061 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:15.515095 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.515219 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.515374 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.515674 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":340,"threads_started":1}
I20260810 18:53:15.517701 27891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: Bootstrap starting.
I20260810 18:53:15.520470 27891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:15.521538 27891 log.cc:824] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:15.523897 27891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: No bootstrap required, opened a new log
I20260810 18:53:15.530774 27891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } }
I20260810 18:53:15.531095 27891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:15.531188 27891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d03f0c0394435e94412f472c70b982, State: Initialized, Role: FOLLOWER
I20260810 18:53:15.531698 27891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } }
I20260810 18:53:15.531908 27891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:15.532027 27891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:15.532182 27891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:15.534263 27891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } }
I20260810 18:53:15.534780 27891 leader_election.cc:302] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49d03f0c0394435e94412f472c70b982; no voters:
I20260810 18:53:15.535279 27891 leader_election.cc:288] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:15.535378 27899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:15.536020 27899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [term 1 LEADER]: Becoming Leader. State: Replica: 49d03f0c0394435e94412f472c70b982, State: Running, Role: LEADER
I20260810 18:53:15.536500 27899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } }
I20260810 18:53:15.537027 27891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:15.539116 27904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d03f0c0394435e94412f472c70b982. Latest consensus state: current_term: 1 leader_uuid: "49d03f0c0394435e94412f472c70b982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } } }
I20260810 18:53:15.539192 27900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49d03f0c0394435e94412f472c70b982" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03f0c0394435e94412f472c70b982" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 34165 } } }
I20260810 18:53:15.539551 27904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:15.540491 27900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:15.541749 27911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:15.545961 27911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:15.554064 27911 catalog_manager.cc:1347] Generated new cluster ID: 35169b52618e4409b79993af9e1e0d94
I20260810 18:53:15.554168 27911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:15.576953 27911 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:15.578629 27911 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:15.590456 27911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: Generated new TSK 0
I20260810 18:53:15.591387 27911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:15.603744 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--builtin_ntp_servers=127.29.174.212:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:15.926456 27930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.926715 27930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.926798 27930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.933578 27930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.933656 27930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.933764 27930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:15.942467 27930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:15.943666 27930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.945488 27930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.957262 27937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.957221 27942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.957310 27940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.958333 27930 server_base.cc:1034] running on GCE node
I20260810 18:53:15.958873 27930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.959671 27930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.960873 27930 hybrid_clock.cc:648] HybridClock initialized: now 1786387995960824 us; error 47 us; skew 500 ppm
I20260810 18:53:15.961150 27930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:15.963143 27930 webserver.cc:460] Webserver started at http://127.29.174.193:40919/ using document root <none> and password file <none>
I20260810 18:53:15.963737 27930 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.963827 27930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.964108 27930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.966758 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "7134fbb2796a4a06a16a91272d4da720"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "de6bb605ab4036d1ed3e613ddd0a6123"
server_key_iv: "d5008280a7bce382c4b5482bb85215bc"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.967471 27930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "7134fbb2796a4a06a16a91272d4da720"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "de6bb605ab4036d1ed3e613ddd0a6123"
server_key_iv: "d5008280a7bce382c4b5482bb85215bc"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.972028 27930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:53:15.975265 27957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.976420 27930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:15.976562 27930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "7134fbb2796a4a06a16a91272d4da720"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-6zbq"
server_key: "de6bb605ab4036d1ed3e613ddd0a6123"
server_key_iv: "d5008280a7bce382c4b5482bb85215bc"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.976760 27930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.992157 27930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.992851 27930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.993119 27930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.993973 27930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:15.995316 27930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:15.995410 27930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.995512 27930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:15.995579 27930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.024385 27930 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:44803
I20260810 18:53:16.024444 28145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:44803 every 8 connection(s)
I20260810 18:53:16.025588 27930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:16.030836 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 27930
I20260810 18:53:16.030985 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:16.031414 30395 external_mini_cluster.cc:1428] Setting key f4419c2f816a1cfbc7144b17f7204b09
I20260810 18:53:16.034113 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--builtin_ntp_servers=127.29.174.212:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:16.037663 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.027934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:55515 (local address 127.29.174.254:34165)
0810 18:53:16.028348 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.028353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.029325 (+ 972us) server_negotiation.cc:407] Connection header received
0810 18:53:16.031088 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.031094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.031169 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.031264 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.032477 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.033482 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.035083 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.035497 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.035604 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.036827 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.036834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.036839 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:16.036859 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.036915 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:16.036928 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.037000 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.037230 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.037325 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":214,"threads_started":1}
I20260810 18:53:16.038592 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.028358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:34165 (local address 127.29.174.193:55515)
0810 18:53:16.029169 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.029203 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.030821 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.031503 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.031515 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.031610 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:16.032273 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.032297 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.033636 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.033640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.034939 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.034963 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.035714 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.035722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.035932 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.036528 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:16.036530 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:16.036564 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:16.036632 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:16.037018 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:16.037030 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:16.037036 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.037177 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.037525 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":250,"threads_started":1}
I20260810 18:53:16.040035 28148 heartbeater.cc:347] Connected to a master server at 127.29.174.254:34165
I20260810 18:53:16.040294 28148 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:16.041126 28148 heartbeater.cc:511] Master 127.29.174.254:34165 requested a full tablet report, sending...
I20260810 18:53:16.043154 27834 ts_manager.cc:194] Registered new tserver with Master: 7134fbb2796a4a06a16a91272d4da720 (127.29.174.193:44803)
I20260810 18:53:16.044564 27834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.193:55515
W20260810 18:53:16.352766 28156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:16.353015 28156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:16.353138 28156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:16.360016 28156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:16.360100 28156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:16.360229 28156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:16.368904 28156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:16.370066 28156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:16.371811 28156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:16.383970 28163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:16.383889 28165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:16.383946 28168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:16.384781 28156 server_base.cc:1034] running on GCE node
I20260810 18:53:16.385329 28156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:16.386186 28156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:16.387403 28156 hybrid_clock.cc:648] HybridClock initialized: now 1786387996387346 us; error 50 us; skew 500 ppm
I20260810 18:53:16.387707 28156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:16.389771 28156 webserver.cc:460] Webserver started at http://127.29.174.194:36285/ using document root <none> and password file <none>
I20260810 18:53:16.390439 28156 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:16.390618 28156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:16.390914 28156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:16.393524 28156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "e79c4657e241405aa06aca8519917f7c"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "f857d724a75736fde6aa1537918899da"
server_key_iv: "1a9d33a4f1ab47c1baef317037f355db"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.394284 28156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "e79c4657e241405aa06aca8519917f7c"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "f857d724a75736fde6aa1537918899da"
server_key_iv: "1a9d33a4f1ab47c1baef317037f355db"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.398864 28156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:53:16.401949 28178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.403134 28156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:16.403277 28156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "e79c4657e241405aa06aca8519917f7c"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "f857d724a75736fde6aa1537918899da"
server_key_iv: "1a9d33a4f1ab47c1baef317037f355db"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.403457 28156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:16.417584 28156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:16.418246 28156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:16.418521 28156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:16.419348 28156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:16.420614 28156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:16.420717 28156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.420818 28156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:16.420887 28156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.449771 28156 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:34437
I20260810 18:53:16.449817 28371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:34437 every 8 connection(s)
I20260810 18:53:16.450975 28156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:16.451637 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 28156
I20260810 18:53:16.451819 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:16.452369 30395 external_mini_cluster.cc:1428] Setting key d27dfd0e8d7d1cd7cc803f1dbba2b3f0
I20260810 18:53:16.455317 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--builtin_ntp_servers=127.29.174.212:32781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:16.462560 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.453369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:55825 (local address 127.29.174.254:34165)
0810 18:53:16.453567 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.453574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.454958 (+ 1384us) server_negotiation.cc:407] Connection header received
0810 18:53:16.456714 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.456720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.456797 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.456880 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.457867 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.458594 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.460051 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.460456 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.460565 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.461782 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.461787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.461790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:16.461807 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.461839 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:16.461850 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.461924 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.462119 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.462217 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 18:53:16.463436 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.453883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:34165 (local address 127.29.174.194:55825)
0810 18:53:16.454814 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.454859 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.456471 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.457096 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.457104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.457182 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:16.457698 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.457715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.458751 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.458755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.459914 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.459926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.460659 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.460667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.460870 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.461500 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:16.461503 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:16.461536 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:16.461602 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:16.461936 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:16.461948 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:16.461955 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.462096 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.462478 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":334,"threads_started":1}
I20260810 18:53:16.464654 28374 heartbeater.cc:347] Connected to a master server at 127.29.174.254:34165
I20260810 18:53:16.464905 28374 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:16.465559 28374 heartbeater.cc:511] Master 127.29.174.254:34165 requested a full tablet report, sending...
I20260810 18:53:16.466825 27834 ts_manager.cc:194] Registered new tserver with Master: e79c4657e241405aa06aca8519917f7c (127.29.174.194:34437)
I20260810 18:53:16.467553 27834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.194:55825
W20260810 18:53:16.771193 28377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:16.771433 28377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:16.771549 28377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:16.778761 28377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:16.778870 28377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:16.779008 28377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:16.787730 28377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:34165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:16.788812 28377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:16.790448 28377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:16.802201 28390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:16.802240 28395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:16.802277 28391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:16.803241 28377 server_base.cc:1034] running on GCE node
I20260810 18:53:16.803805 28377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:16.804661 28377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:16.805866 28377 hybrid_clock.cc:648] HybridClock initialized: now 1786387996805816 us; error 44 us; skew 500 ppm
I20260810 18:53:16.806155 28377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:16.808230 28377 webserver.cc:460] Webserver started at http://127.29.174.195:42783/ using document root <none> and password file <none>
I20260810 18:53:16.808831 28377 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:16.808928 28377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:16.809218 28377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:16.811866 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "5984c5e9773c4c79b5888023c795c491"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "75b7fb76ee992231bbca69005120c054"
server_key_iv: "a13751a621cc588d47fa7372335e1d32"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.812602 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "5984c5e9773c4c79b5888023c795c491"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "75b7fb76ee992231bbca69005120c054"
server_key_iv: "a13751a621cc588d47fa7372335e1d32"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.817523 28377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:53:16.820761 28407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.821954 28377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:16.822105 28377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "5984c5e9773c4c79b5888023c795c491"
format_stamp: "Formatted at 2026-08-10 18:53:16 on dist-test-slave-6zbq"
server_key: "75b7fb76ee992231bbca69005120c054"
server_key_iv: "a13751a621cc588d47fa7372335e1d32"
server_key_version: "encryptionkey@0"
I20260810 18:53:16.822299 28377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:16.838800 28377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:16.839501 28377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:16.839766 28377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:16.840601 28377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:16.842213 28377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:16.842311 28377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.842429 28377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:16.842504 28377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:16.872110 28377 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:38773
I20260810 18:53:16.872201 28606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:38773 every 8 connection(s)
I20260810 18:53:16.873395 28377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:16.881888 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 28377
I20260810 18:53:16.882067 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:16.882644 30395 external_mini_cluster.cc:1428] Setting key 5f9dd15cc4b3081b91e0432a7b0aea7e
I20260810 18:53:16.885179 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.875632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:52583 (local address 127.29.174.254:34165)
0810 18:53:16.875801 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.875805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.877022 (+ 1217us) server_negotiation.cc:407] Connection header received
0810 18:53:16.878669 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.878673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.878741 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.878827 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.879854 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.880637 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.882163 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.882852 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.882983 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.884378 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.884384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.884387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:16.884405 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.884442 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:16.884457 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.884550 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.884737 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.884835 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:53:16.885947 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.876061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:34165 (local address 127.29.174.195:52583)
0810 18:53:16.876885 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.876918 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.878448 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.878995 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.879003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.879080 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:16.879647 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.879666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.880789 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.880793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.881970 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.881982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.883056 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.883060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.883345 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.884024 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:16.884027 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:16.884098 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:16.884193 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:16.884555 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:16.884563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:16.884569 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.884713 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.884969 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":248,"threads_started":1}
I20260810 18:53:16.887102 28607 heartbeater.cc:347] Connected to a master server at 127.29.174.254:34165
I20260810 18:53:16.887341 28607 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:16.887977 28607 heartbeater.cc:511] Master 127.29.174.254:34165 requested a full tablet report, sending...
I20260810 18:53:16.889160 27834 ts_manager.cc:194] Registered new tserver with Master: 5984c5e9773c4c79b5888023c795c491 (127.29.174.195:38773)
I20260810 18:53:16.889876 27834 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.195:52583
I20260810 18:53:16.897336 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:16.906332 28153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.901311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57070 (local address 127.29.174.254:34165)
0810 18:53:16.901511 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.901517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.901830 (+ 313us) server_negotiation.cc:407] Connection header received
0810 18:53:16.901949 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.901952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.902018 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.902108 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.902612 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.903316 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.904535 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.904905 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.904995 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.905809 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.905813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.905816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:16.905832 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.905862 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:16.905872 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.905942 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.906022 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.906115 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:53:16.909646 27834 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:16.916105 27834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:16.936686 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.927278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:34437 (local address 127.0.0.1:50136)
0810 18:53:16.929416 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.929437 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.929558 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.930091 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.930096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.930119 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:16.930473 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.930486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.932469 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.932474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.933979 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.933992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.935741 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.935748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.935968 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.936020 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.936156 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.936336 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"mutex_wait_us":56,"thread_start_us":1840,"threads_started":1}
I20260810 18:53:16.937263 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.926597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47744)
0810 18:53:16.928616 (+ 2019us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.928658 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.928930 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.929714 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.929720 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.929755 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:16.930039 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.930050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.932201 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.932210 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.934531 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.934546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.936248 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.936254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.936407 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.936440 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.936522 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.936643 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":29,"thread_start_us":350,"threads_started":1}
I20260810 18:53:16.937379 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.926799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.29.174.193:44803)
0810 18:53:16.927594 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.927602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.928804 (+ 1202us) server_negotiation.cc:407] Connection header received
0810 18:53:16.929139 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.929147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.929382 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.929565 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:16.930262 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.932078 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.934780 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.936033 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.936201 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.936385 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.936582 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.936756 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":919,"threads_started":1}
I20260810 18:53:16.937508 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.927545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.29.174.194:34437)
0810 18:53:16.928067 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.928074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.929546 (+ 1472us) server_negotiation.cc:407] Connection header received
0810 18:53:16.929693 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.929700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.929904 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.930092 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:16.930650 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.932343 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.934151 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.935538 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.935729 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.935934 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.936964 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.937169 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":652,"threads_started":1}
I20260810 18:53:16.940725 28271 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77c670db824798a111729624e8cc01 (DEFAULT_TABLE table=test-table [id=d6c7e028ad0c4c279ae60f1a0c44b3f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:16.940733 28033 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77c670db824798a111729624e8cc01 (DEFAULT_TABLE table=test-table [id=d6c7e028ad0c4c279ae60f1a0c44b3f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:16.942046 28271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77c670db824798a111729624e8cc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:16.942070 28033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77c670db824798a111729624e8cc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:16.942299 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.927722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:38773 (local address 127.0.0.1:45756)
0810 18:53:16.931181 (+ 3459us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.931198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.931300 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.931906 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.931911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.931934 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:16.932322 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.932333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.934185 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.934190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.936165 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.936177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.941546 (+ 5369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.941554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.941768 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.941801 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.941893 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.942017 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3363,"thread_start_us":3231,"threads_started":1}
I20260810 18:53:16.942494 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.928268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.29.174.195:38773)
0810 18:53:16.930805 (+ 2537us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.930813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.931412 (+ 599us) server_negotiation.cc:407] Connection header received
0810 18:53:16.931498 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.931504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.931723 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.931906 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:16.932547 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.934041 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.936391 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.941344 (+ 4953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.941555 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.941745 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.941965 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.942117 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2375,"thread_start_us":2224,"threads_started":1}
I20260810 18:53:16.945953 28485 tablet_service.cc:1467] Processing CreateTablet for tablet 6a77c670db824798a111729624e8cc01 (DEFAULT_TABLE table=test-table [id=d6c7e028ad0c4c279ae60f1a0c44b3f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:16.947417 28485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a77c670db824798a111729624e8cc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:16.953212 28636 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Bootstrap starting.
I20260810 18:53:16.956642 28637 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Bootstrap starting.
I20260810 18:53:16.957216 28636 tablet_bootstrap.cc:654] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:16.958521 28636 log.cc:824] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:16.959950 28637 tablet_bootstrap.cc:654] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:16.960274 28641 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Bootstrap starting.
I20260810 18:53:16.961274 28637 log.cc:824] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:16.961480 28636 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: No bootstrap required, opened a new log
I20260810 18:53:16.961724 28636 ts_tablet_manager.cc:1397] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 18:53:16.963408 28641 tablet_bootstrap.cc:654] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:16.964332 28637 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: No bootstrap required, opened a new log
I20260810 18:53:16.964545 28641 log.cc:824] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:16.964649 28637 ts_tablet_manager.cc:1397] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 18:53:16.969154 28636 raft_consensus.cc:348] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.969482 28636 raft_consensus.cc:374] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:16.969640 28636 raft_consensus.cc:729] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7134fbb2796a4a06a16a91272d4da720, State: Initialized, Role: FOLLOWER
I20260810 18:53:16.970309 28636 consensus_queue.cc:260] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.971380 28148 heartbeater.cc:503] Master 127.29.174.254:34165 was elected leader, sending a full tablet report...
I20260810 18:53:16.971837 28636 ts_tablet_manager.cc:1428] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260810 18:53:16.972715 28641 tablet_bootstrap.cc:492] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: No bootstrap required, opened a new log
I20260810 18:53:16.972957 28641 ts_tablet_manager.cc:1397] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260810 18:53:16.974476 28637 raft_consensus.cc:348] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.974913 28637 raft_consensus.cc:374] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:16.975015 28637 raft_consensus.cc:729] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e79c4657e241405aa06aca8519917f7c, State: Initialized, Role: FOLLOWER
I20260810 18:53:16.975606 28637 consensus_queue.cc:260] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.978884 28374 heartbeater.cc:503] Master 127.29.174.254:34165 was elected leader, sending a full tablet report...
I20260810 18:53:16.979602 28637 ts_tablet_manager.cc:1428] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 18:53:16.981989 28641 raft_consensus.cc:348] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.982303 28641 raft_consensus.cc:374] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:16.982429 28641 raft_consensus.cc:729] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5984c5e9773c4c79b5888023c795c491, State: Initialized, Role: FOLLOWER
I20260810 18:53:16.982919 28641 consensus_queue.cc:260] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:16.983812 28607 heartbeater.cc:503] Master 127.29.174.254:34165 was elected leader, sending a full tablet report...
I20260810 18:53:16.984253 28641 ts_tablet_manager.cc:1428] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
W20260810 18:53:17.027578 28149 tablet.cc:2369] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:17.124997 28608 tablet.cc:2369] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:17.174373 28649 raft_consensus.cc:482] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:17.174654 28649 raft_consensus.cc:504] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:17.176254 28649 leader_election.cc:288] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7134fbb2796a4a06a16a91272d4da720 (127.29.174.193:44803), 5984c5e9773c4c79b5888023c795c491 (127.29.174.195:38773)
I20260810 18:53:17.178180 28647 raft_consensus.cc:482] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:17.178601 28647 raft_consensus.cc:504] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:17.184921 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.177367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:44073 (local address 127.29.174.195:38773)
0810 18:53:17.177539 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.177545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.177984 (+ 439us) server_negotiation.cc:407] Connection header received
0810 18:53:17.178155 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.178160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.178216 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.178287 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:17.178888 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.179899 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.182372 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.183725 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.183859 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.183932 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.184476 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.184620 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:17.185699 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.177239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:38773 (local address 127.29.174.194:44073)
0810 18:53:17.177874 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.177888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:17.178004 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:17.178370 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:17.178374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:17.178398 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:17.178718 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.178732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.180051 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.180060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.182150 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.182167 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.184083 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.184089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.184286 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.184322 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:17.185202 (+ 880us) client_negotiation.cc:240] Negotiation successful
0810 18:53:17.185348 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":2,"thread_start_us":353,"threads_started":1}
I20260810 18:53:17.186858 28647 leader_election.cc:288] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5984c5e9773c4c79b5888023c795c491 (127.29.174.195:38773), e79c4657e241405aa06aca8519917f7c (127.29.174.194:34437)
I20260810 18:53:17.186812 28530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "e79c4657e241405aa06aca8519917f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5984c5e9773c4c79b5888023c795c491" is_pre_election: true
I20260810 18:53:17.187439 28530 raft_consensus.cc:2454] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e79c4657e241405aa06aca8519917f7c in term 0.
I20260810 18:53:17.188272 28193 leader_election.cc:302] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5984c5e9773c4c79b5888023c795c491, e79c4657e241405aa06aca8519917f7c; no voters:
I20260810 18:53:17.188630 28649 raft_consensus.cc:2781] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:17.188834 28649 raft_consensus.cc:482] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:17.189060 28649 raft_consensus.cc:3037] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:17.189980 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.176738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:45597 (local address 127.29.174.193:44803)
0810 18:53:17.176916 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.176921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.177267 (+ 346us) server_negotiation.cc:407] Connection header received
0810 18:53:17.177481 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.177487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.177580 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.177684 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:17.178406 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.179448 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.187253 (+ 7805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.188566 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.188691 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.188758 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.189440 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.189678 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:17.190838 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.176523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.29.174.194:45597)
0810 18:53:17.177163 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.177187 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:17.177329 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:17.177722 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:17.177727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:17.177758 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:17.178191 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.178206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.179593 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.179597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.187016 (+ 7419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.187030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.188961 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.188968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.189219 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.189262 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:53:17.190133 (+ 871us) client_negotiation.cc:240] Negotiation successful
0810 18:53:17.190302 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":33,"thread_start_us":314,"threads_started":1}
I20260810 18:53:17.192313 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "e79c4657e241405aa06aca8519917f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7134fbb2796a4a06a16a91272d4da720" is_pre_election: true
I20260810 18:53:17.192759 28649 raft_consensus.cc:504] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:17.192791 28065 raft_consensus.cc:2454] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e79c4657e241405aa06aca8519917f7c in term 0.
I20260810 18:53:17.193739 28065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "e79c4657e241405aa06aca8519917f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7134fbb2796a4a06a16a91272d4da720"
I20260810 18:53:17.193953 28065 raft_consensus.cc:3037] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:17.194089 28649 leader_election.cc:288] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [CANDIDATE]: Term 1 election: Requested vote from peers 7134fbb2796a4a06a16a91272d4da720 (127.29.174.193:44803), 5984c5e9773c4c79b5888023c795c491 (127.29.174.195:38773)
I20260810 18:53:17.195021 28530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "e79c4657e241405aa06aca8519917f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5984c5e9773c4c79b5888023c795c491"
I20260810 18:53:17.195255 28530 raft_consensus.cc:3037] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:17.197770 28065 raft_consensus.cc:2454] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e79c4657e241405aa06aca8519917f7c in term 1.
I20260810 18:53:17.198137 28193 leader_election.cc:302] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7134fbb2796a4a06a16a91272d4da720, e79c4657e241405aa06aca8519917f7c; no voters:
I20260810 18:53:17.198318 28530 raft_consensus.cc:2454] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e79c4657e241405aa06aca8519917f7c in term 1.
I20260810 18:53:17.198745 28649 raft_consensus.cc:2781] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:17.199298 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.181208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:48665 (local address 127.29.174.195:38773)
0810 18:53:17.185156 (+ 3948us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.185162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.191877 (+ 6715us) server_negotiation.cc:407] Connection header received
0810 18:53:17.192176 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.192186 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.192275 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.192335 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:17.193142 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.194164 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.196928 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.198234 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.198389 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.198453 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.198957 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.199100 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3843,"thread_start_us":10008,"threads_started":1}
I20260810 18:53:17.199770 28649 raft_consensus.cc:686] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 1 LEADER]: Becoming Leader. State: Replica: e79c4657e241405aa06aca8519917f7c, State: Running, Role: LEADER
I20260810 18:53:17.199977 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.180671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:38773 (local address 127.29.174.193:48665)
0810 18:53:17.191770 (+ 11099us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.191789 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:17.191984 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:17.192521 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:17.192527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:17.192557 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:17.192959 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.192970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.194374 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.194380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.196683 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.196697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.198630 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.198637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.198788 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.198820 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:53:17.199496 (+ 676us) client_negotiation.cc:240] Negotiation successful
0810 18:53:17.199675 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10954,"thread_start_us":10761,"threads_started":1}
I20260810 18:53:17.200489 28649 consensus_queue.cc:237] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } }
I20260810 18:53:17.200641 28530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "7134fbb2796a4a06a16a91272d4da720" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5984c5e9773c4c79b5888023c795c491" is_pre_election: true
I20260810 18:53:17.200903 28530 raft_consensus.cc:2379] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7134fbb2796a4a06a16a91272d4da720 in current term 1: Already voted for candidate e79c4657e241405aa06aca8519917f7c in this term.
I20260810 18:53:17.201004 28662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.180815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:34437 (local address 127.29.174.193:40323)
0810 18:53:17.193948 (+ 13133us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.193970 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:17.194085 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:17.194452 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:17.194456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:17.194488 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:17.194985 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.195006 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.196364 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.196370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.198438 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:17.198453 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.200298 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.200305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:17.200462 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.200496 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:53:17.200588 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:53:17.200711 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13002,"thread_start_us":13342,"threads_started":1}
I20260810 18:53:17.201274 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.181245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:40323 (local address 127.29.174.194:34437)
0810 18:53:17.181424 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.181430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.194038 (+ 12608us) server_negotiation.cc:407] Connection header received
0810 18:53:17.194187 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.194192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.194256 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.194358 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:17.195207 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.196191 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.198655 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.199972 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.200113 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.200170 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.200915 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.201038 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:17.202016 28305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a77c670db824798a111729624e8cc01" candidate_uuid: "7134fbb2796a4a06a16a91272d4da720" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e79c4657e241405aa06aca8519917f7c" is_pre_election: true
W20260810 18:53:17.202884 28375 tablet.cc:2369] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:17.202936 27959 leader_election.cc:302] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7134fbb2796a4a06a16a91272d4da720; no voters: 5984c5e9773c4c79b5888023c795c491, e79c4657e241405aa06aca8519917f7c
I20260810 18:53:17.204757 28647 raft_consensus.cc:2726] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 18:53:17.205760 27833 catalog_manager.cc:5495] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c reported cstate change: term changed from 0 to 1, leader changed from <none> to e79c4657e241405aa06aca8519917f7c (127.29.174.194). New cstate: current_term: 1 leader_uuid: "e79c4657e241405aa06aca8519917f7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e79c4657e241405aa06aca8519917f7c" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 34437 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:17.226867 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.222004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.29.174.194:34437)
0810 18:53:17.222135 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.222139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.222315 (+ 176us) server_negotiation.cc:407] Connection header received
0810 18:53:17.222505 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.222509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.222593 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.222711 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:17.222981 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.223652 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.224993 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.225400 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.225495 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.225944 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:17.226417 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:17.226551 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.226590 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.226645 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:53:17.233724 28530 raft_consensus.cc:1261] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 1 FOLLOWER]: Refusing update from remote peer e79c4657e241405aa06aca8519917f7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:17.233788 28065 raft_consensus.cc:1261] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 1 FOLLOWER]: Refusing update from remote peer e79c4657e241405aa06aca8519917f7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:17.234476 28665 consensus_queue.cc:1035] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5984c5e9773c4c79b5888023c795c491" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 38773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:17.234798 28649 consensus_queue.cc:1035] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7134fbb2796a4a06a16a91272d4da720" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:17.245137 28671 mvcc.cc:205] Tried to move back new op lower bound from 7317045236661223424 to 7317045236539129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045236661223424}]
I20260810 18:53:17.245208 28673 mvcc.cc:205] Tried to move back new op lower bound from 7317045236661223424 to 7317045236539129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045236661223424}]
I20260810 18:53:17.250083 28672 mvcc.cc:205] Tried to move back new op lower bound from 7317045236661223424 to 7317045236539129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045236661223424}]
I20260810 18:53:17.256832 27833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:17.257048 27833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:17.260277 27833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49d03f0c0394435e94412f472c70b982: Sending DeleteTablet for 3 replicas of tablet 6a77c670db824798a111729624e8cc01
I20260810 18:53:17.261281 28033 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a77c670db824798a111729624e8cc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:17 UTC) from {username='slave'} at 127.0.0.1:47744
I20260810 18:53:17.261293 28271 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a77c670db824798a111729624e8cc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:17 UTC) from {username='slave'} at 127.0.0.1:50136
I20260810 18:53:17.261291 28485 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a77c670db824798a111729624e8cc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:17 UTC) from {username='slave'} at 127.0.0.1:45756
I20260810 18:53:17.262591 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 27930
I20260810 18:53:17.276445 28686 tablet_replica.cc:330] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: stopping tablet replica
I20260810 18:53:17.277361 28686 raft_consensus.cc:2229] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:17.277559 28685 tablet_replica.cc:330] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: stopping tablet replica
I20260810 18:53:17.277938 28685 raft_consensus.cc:2229] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:17.277999 28686 raft_consensus.cc:2258] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:17.279943 28685 raft_consensus.cc:2258] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:17.280331 28686 ts_tablet_manager.cc:1905] T 6a77c670db824798a111729624e8cc01 P 7134fbb2796a4a06a16a91272d4da720: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:17.280416 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.263659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.29.174.193:44803)
0810 18:53:17.263839 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.263845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.271245 (+ 7400us) server_negotiation.cc:407] Connection header received
0810 18:53:17.272038 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.272045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.272140 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.272210 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:17.273424 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.274707 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.276629 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.277311 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.277414 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.278850 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:17.278861 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:17.278875 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:17.279525 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:17.279581 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:17.279605 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:17.279697 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.279994 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.280094 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:17.281530 28141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:17.283725 28685 ts_tablet_manager.cc:1905] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:17.283993 28682 tablet_replica.cc:330] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: stopping tablet replica
I20260810 18:53:17.284329 28682 raft_consensus.cc:2229] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:17.284754 28682 raft_consensus.cc:2258] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:17.290838 28685 ts_tablet_manager.cc:1918] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:17.290977 28685 log.cc:1196] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/6a77c670db824798a111729624e8cc01
I20260810 18:53:17.291410 28685 ts_tablet_manager.cc:1939] T 6a77c670db824798a111729624e8cc01 P e79c4657e241405aa06aca8519917f7c: Deleting consensus metadata
I20260810 18:53:17.292548 28682 ts_tablet_manager.cc:1905] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:17.292761 27808 catalog_manager.cc:4841] TS e79c4657e241405aa06aca8519917f7c (127.29.174.194:34437): tablet 6a77c670db824798a111729624e8cc01 (table test-table [id=d6c7e028ad0c4c279ae60f1a0c44b3f6]) successfully deleted
I20260810 18:53:17.296998 28682 ts_tablet_manager.cc:1918] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:17.297133 28682 log.cc:1196] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/6a77c670db824798a111729624e8cc01
I20260810 18:53:17.297521 28682 ts_tablet_manager.cc:1939] T 6a77c670db824798a111729624e8cc01 P 5984c5e9773c4c79b5888023c795c491: Deleting consensus metadata
I20260810 18:53:17.298770 27815 catalog_manager.cc:4841] TS 5984c5e9773c4c79b5888023c795c491 (127.29.174.195:38773): tablet 6a77c670db824798a111729624e8cc01 (table test-table [id=d6c7e028ad0c4c279ae60f1a0c44b3f6]) successfully deleted
I20260810 18:53:17.433959 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 27930
W20260810 18:53:17.456152 27815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:44803 (error 108)
I20260810 18:53:17.457952 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 28156
I20260810 18:53:17.458010 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.457268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47762)
0810 18:53:17.457689 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.457823 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":199,"threads_started":1}
W20260810 18:53:17.458256 27815 catalog_manager.cc:4584] TS 7134fbb2796a4a06a16a91272d4da720 (127.29.174.193:44803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a77c670db824798a111729624e8cc01: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
I20260810 18:53:17.465009 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.458957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.29.174.194:34437)
0810 18:53:17.459113 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.459118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.459137 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:17.459275 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.459280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.459390 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.459482 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:17.459906 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.460692 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.461942 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.462489 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.462614 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.463674 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:17.463683 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:17.463696 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:17.464265 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:17.464312 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:17.464333 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:17.464443 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.464574 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.464686 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:17.465669 28362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:17.511945 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.511449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47778)
0810 18:53:17.511682 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.511788 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:53:17.513044 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.512680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47786)
0810 18:53:17.512834 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.512920 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:53:17.573709 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.573012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47798)
0810 18:53:17.573410 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.573508 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253}
I20260810 18:53:17.573662 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 28156
I20260810 18:53:17.583537 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.583085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47808)
0810 18:53:17.583262 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.583356 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:53:17.598307 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 28377
I20260810 18:53:17.604986 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.599041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.29.174.195:38773)
0810 18:53:17.599216 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.599222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.599241 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:17.599423 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.599428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.599512 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.599597 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:17.600005 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.600713 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.602029 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.602563 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.602676 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.603747 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:17.603755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:17.603771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:17.604312 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:17.604357 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:17.604376 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:17.604472 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.604600 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.604694 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:17.605561 28595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:17.664198 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.663642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47818)
0810 18:53:17.663880 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.664006 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 18:53:17.665297 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.664824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:44803 (local address 127.0.0.1:47832)
0810 18:53:17.665019 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:17.665112 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:44803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 18:53:17.692402 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 28377
I20260810 18:53:17.712050 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 27782
I20260810 18:53:17.713091 27885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:17.818679 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 27782
2026-08-10T18:53:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2812 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:17 dist-test-slave-6zbq krb5kdc[28702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:17 dist-test-slave-6zbq krb5kdc[28702](info): set up 2 sockets
Aug 10 18:53:17 dist-test-slave-6zbq krb5kdc[28702](info): commencing operation
krb5kdc: starting...
W20260810 18:53:19.051116 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:19 dist-test-slave-6zbq krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:53:19.311355 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43333
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:43333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:19.630918 28720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:19.631179 28720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:19.631304 28720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:19.638173 28720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:19.638263 28720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:19.638370 28720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:19.638459 28720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:19.638518 28720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:19.647755 28720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:43333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:43333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:19.648921 28720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:19.650156 28720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:19.661103 28730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:19.661067 28733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:19.661028 28731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:19.662081 28720 server_base.cc:1034] running on GCE node
I20260810 18:53:19.662822 28720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:19.664088 28720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:19.665282 28720 hybrid_clock.cc:648] HybridClock initialized: now 1786387999665239 us; error 27 us; skew 500 ppm
I20260810 18:53:19.665583 28720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:19 dist-test-slave-6zbq krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:19.668404 28720 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:53:19.669742 28720 webserver.cc:460] Webserver started at http://127.29.174.254:38383/ using document root <none> and password file <none>
I20260810 18:53:19.670370 28720 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:19.670497 28720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:19.670779 28720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:19.673532 28720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "8e6f72445370407a85b9d51cb781e2a9"
format_stamp: "Formatted at 2026-08-10 18:53:19 on dist-test-slave-6zbq"
server_key: "ed00868af46291b61c4e764c30e14430"
server_key_iv: "c570510cefc40eed4b213fe31d3bbf77"
server_key_version: "encryptionkey@0"
I20260810 18:53:19.674237 28720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "8e6f72445370407a85b9d51cb781e2a9"
format_stamp: "Formatted at 2026-08-10 18:53:19 on dist-test-slave-6zbq"
server_key: "ed00868af46291b61c4e764c30e14430"
server_key_iv: "c570510cefc40eed4b213fe31d3bbf77"
server_key_version: "encryptionkey@0"
I20260810 18:53:19.678912 28720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:53:19.682014 28745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:19.683327 28720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:53:19.683486 28720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "8e6f72445370407a85b9d51cb781e2a9"
format_stamp: "Formatted at 2026-08-10 18:53:19 on dist-test-slave-6zbq"
server_key: "ed00868af46291b61c4e764c30e14430"
server_key_iv: "c570510cefc40eed4b213fe31d3bbf77"
server_key_version: "encryptionkey@0"
I20260810 18:53:19.683651 28720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:19.708247 28720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:19.712054 28720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:19.712306 28720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:19.729648 28720 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:43333
I20260810 18:53:19.729665 28829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:43333 every 8 connection(s)
I20260810 18:53:19.730957 28720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:19.733734 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 28720
I20260810 18:53:19.733920 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:19.734548 30395 external_mini_cluster.cc:1428] Setting key c72aaca0de48bb9c36645c661acb6e1a
I20260810 18:53:19.734728 28832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.746481 28832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: Bootstrap starting.
Aug 10 18:53:19 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:19.749578 28832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.750814 28832 log.cc:824] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:19.753342 28832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: No bootstrap required, opened a new log
I20260810 18:53:19.755924 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.736342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.29.174.254:43333)
0810 18:53:19.737077 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.737088 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.737133 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:53:19.739220 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.739240 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.739583 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.739972 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:19.741408 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.742722 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.743996 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.744693 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.744865 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.749052 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:19.749059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:19.749072 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:19.749107 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:19.752129 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.752807 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.752813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.752819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.752976 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.753348 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.753351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.753354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.753850 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:19.754137 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.754606 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.755002 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":317,"threads_started":1}
I20260810 18:53:19.761019 28832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } }
I20260810 18:53:19.761350 28832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.761438 28832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6f72445370407a85b9d51cb781e2a9, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.761924 28832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } }
I20260810 18:53:19.762135 28832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:19.762255 28832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:19.762413 28832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.764441 28832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } }
I20260810 18:53:19.764884 28832 leader_election.cc:302] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e6f72445370407a85b9d51cb781e2a9; no voters:
I20260810 18:53:19.765559 28832 leader_election.cc:288] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:19.765671 28844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:19.766278 28844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [term 1 LEADER]: Becoming Leader. State: Replica: 8e6f72445370407a85b9d51cb781e2a9, State: Running, Role: LEADER
I20260810 18:53:19.766829 28844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } }
I20260810 18:53:19.767656 28832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:19.769447 28846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e6f72445370407a85b9d51cb781e2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } } }
I20260810 18:53:19.769806 28846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:19.770547 28849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e6f72445370407a85b9d51cb781e2a9. Latest consensus state: current_term: 1 leader_uuid: "8e6f72445370407a85b9d51cb781e2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6f72445370407a85b9d51cb781e2a9" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 43333 } } }
I20260810 18:53:19.770912 28849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:19.775298 28858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:19.780344 28858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:19.788897 28858 catalog_manager.cc:1347] Generated new cluster ID: 155ff217cdae4498b9d5eced8e2d4174
I20260810 18:53:19.788992 28858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:19.801996 28858 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:19.803194 28858 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:19.812867 28858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: Generated new TSK 0
I20260810 18:53:19.813525 28858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:53:19.890027 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:20.233696 28891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:20.233994 28891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:20.234136 28891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:20.241586 28891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:20.241729 28891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:20.241881 28891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:20.252140 28891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:20.253551 28891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:20.255026 28891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:20.268945 28900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:20.268894 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:20.269447 28899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:20.269825 28891 server_base.cc:1034] running on GCE node
I20260810 18:53:20.270560 28891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:20.271639 28891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:20.272858 28891 hybrid_clock.cc:648] HybridClock initialized: now 1786388000272798 us; error 48 us; skew 500 ppm
I20260810 18:53:20.273203 28891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:20 dist-test-slave-6zbq krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388000, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:20.276798 28891 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:53:20.278784 28891 webserver.cc:460] Webserver started at http://127.29.174.193:44609/ using document root <none> and password file <none>
I20260810 18:53:20.279551 28891 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:20.279690 28891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:20.280035 28891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:20.283424 28891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "fd57474b4ed24044ad292449a1ba71bf"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "add557de40293086ac1cc82f288a6c20"
server_key_iv: "6c2b18f47f2e5507658d33d299478d6a"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.284305 28891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "fd57474b4ed24044ad292449a1ba71bf"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "add557de40293086ac1cc82f288a6c20"
server_key_iv: "6c2b18f47f2e5507658d33d299478d6a"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.290282 28891 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:53:20.294250 28914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.296640 28891 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:20.296955 28891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "fd57474b4ed24044ad292449a1ba71bf"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "add557de40293086ac1cc82f288a6c20"
server_key_iv: "6c2b18f47f2e5507658d33d299478d6a"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.297276 28891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:20.326754 28891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:20.333514 28891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:20.333848 28891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:20.335080 28891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:20.336560 28891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:20.336638 28891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.336746 28891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:20.336812 28891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.371080 28891 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35031
I20260810 18:53:20.371133 29106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35031 every 8 connection(s)
I20260810 18:53:20.372473 28891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:20.373850 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 28891
I20260810 18:53:20.374058 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:20.374686 30395 external_mini_cluster.cc:1428] Setting key 87ff7df46a031aac8636e20502a0460a
Aug 10 18:53:20 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388000, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:20.397042 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:20.375771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:55431 (local address 127.29.174.254:43333)
0810 18:53:20.376327 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:20.376333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:20.377431 (+ 1098us) server_negotiation.cc:407] Connection header received
0810 18:53:20.379736 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:20.379741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:20.379845 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:20.379943 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:20.381639 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.382479 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.384127 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.384551 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.384684 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:20.389374 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:20.389381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:20.389384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:20.389431 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:20.392789 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:20.393762 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:20.393770 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:20.393773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:20.393972 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:20.394452 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:20.394457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:20.394460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:20.394816 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:20.394968 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:20.396374 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0810 18:53:20.396673 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":259,"threads_started":1}
I20260810 18:53:20.398527 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:20.376451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43333 (local address 127.29.174.193:55431)
0810 18:53:20.377272 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:20.377321 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:20.379434 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:20.380229 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:20.380238 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:20.380797 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:20.381450 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:20.381471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.382647 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.382651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:20.383957 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:20.383978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.384810 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.384818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:20.385330 (+ 512us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:20.386450 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:20.386482 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:20.389093 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:20.393040 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:20.393050 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:20.393068 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:20.393593 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:20.394082 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:20.394085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:20.394088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:20.394290 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:20.395113 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:20.395125 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:20.395742 (+ 617us) client_negotiation.cc:769] Sending connection context
0810 18:53:20.396162 (+ 420us) client_negotiation.cc:240] Negotiation successful
0810 18:53:20.396804 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":345,"threads_started":1}
I20260810 18:53:20.401019 29107 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43333
I20260810 18:53:20.401757 29107 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:20.403460 29107 heartbeater.cc:511] Master 127.29.174.254:43333 requested a full tablet report, sending...
I20260810 18:53:20.406551 28768 ts_manager.cc:194] Registered new tserver with Master: fd57474b4ed24044ad292449a1ba71bf (127.29.174.193:35031)
I20260810 18:53:20.408435 28768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:55431
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:53:20.444950 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:20.758651 29118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:20.758898 29118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:20.759025 29118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:20.765863 29118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:20.765966 29118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:20.766139 29118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:20.775343 29118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:20.776444 29118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:20.777619 29118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:20.789439 29131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:20.789480 29127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:20.789503 29129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:20.790462 29118 server_base.cc:1034] running on GCE node
I20260810 18:53:20.790992 29118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:20.791798 29118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:20.792989 29118 hybrid_clock.cc:648] HybridClock initialized: now 1786388000792940 us; error 41 us; skew 500 ppm
I20260810 18:53:20.793262 29118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:20 dist-test-slave-6zbq krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388000, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:20.799525 29118 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:53:20.800745 29118 webserver.cc:460] Webserver started at http://127.29.174.194:40963/ using document root <none> and password file <none>
I20260810 18:53:20.801327 29118 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:20.801420 29118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:20.801668 29118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:20.804334 29118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "83a56d27af8f433ea779c70ff968845f"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "5b4c895d9001ba8bdd70b3dbdf02a158"
server_key_iv: "cd5fcb5f280f622978d1bbf27e43ccec"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.805011 29118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "83a56d27af8f433ea779c70ff968845f"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "5b4c895d9001ba8bdd70b3dbdf02a158"
server_key_iv: "cd5fcb5f280f622978d1bbf27e43ccec"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.809650 29118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:20.812726 29143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.813952 29118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:20.814072 29118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "83a56d27af8f433ea779c70ff968845f"
format_stamp: "Formatted at 2026-08-10 18:53:20 on dist-test-slave-6zbq"
server_key: "5b4c895d9001ba8bdd70b3dbdf02a158"
server_key_iv: "cd5fcb5f280f622978d1bbf27e43ccec"
server_key_version: "encryptionkey@0"
I20260810 18:53:20.814225 29118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:20.844441 29118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:20.848299 29118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:20.848526 29118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:20.849363 29118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:20.850577 29118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:20.850647 29118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.850749 29118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:20.850791 29118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:20.878149 29118 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:43353
I20260810 18:53:20.878196 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:43353 every 8 connection(s)
I20260810 18:53:20.879297 29118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:20.888792 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 29118
I20260810 18:53:20.888962 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:20.889387 30395 external_mini_cluster.cc:1428] Setting key 7166a377ba2b90a1f75a99f1f5288b72
Aug 10 18:53:20 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388000, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:20.897634 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:20.881549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:43475 (local address 127.29.174.254:43333)
0810 18:53:20.881724 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:20.881729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:20.882792 (+ 1063us) server_negotiation.cc:407] Connection header received
0810 18:53:20.884519 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:20.884525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:20.884602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:20.884692 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:20.886045 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.886836 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.888355 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.888774 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.888883 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:20.892430 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:20.892435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:20.892439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:20.892470 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:20.894857 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:20.895520 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:20.895526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:20.895529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:20.895650 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:20.896052 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:20.896057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:20.896059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:20.896314 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:20.896449 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:20.897084 (+ 635us) server_negotiation.cc:299] Negotiation successful
0810 18:53:20.897310 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:53:20.898555 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:20.881999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43333 (local address 127.29.174.194:43475)
0810 18:53:20.882658 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:20.882693 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:20.884274 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:20.884909 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:20.884917 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:20.885374 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:20.885884 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:20.885901 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.886986 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.886991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:20.888240 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:20.888253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:20.888987 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:20.888994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:20.889208 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:20.889831 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:20.889860 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:20.892217 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:20.894986 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:20.894994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:20.895008 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:20.895408 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:20.895758 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:20.895763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:20.895767 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:20.895944 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:20.896438 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:20.896445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:20.896749 (+ 304us) client_negotiation.cc:769] Sending connection context
0810 18:53:20.896977 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 18:53:20.897449 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":289,"threads_started":1}
I20260810 18:53:20.899883 29343 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43333
I20260810 18:53:20.900136 29343 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:20.900774 29343 heartbeater.cc:511] Master 127.29.174.254:43333 requested a full tablet report, sending...
I20260810 18:53:20.902230 28768 ts_manager.cc:194] Registered new tserver with Master: 83a56d27af8f433ea779c70ff968845f (127.29.174.194:43353)
I20260810 18:53:20.903227 28768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:43475
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:53:20.948518 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:21.258114 29350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:21.258389 29350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:21.258523 29350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:21.265606 29350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:21.265717 29350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:21.265852 29350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:21.275130 29350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:43333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:21.276207 29350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:21.277384 29350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:21.289214 29361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.289301 29358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.289417 29367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:21.290249 29350 server_base.cc:1034] running on GCE node
I20260810 18:53:21.290899 29350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:21.291710 29350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:21.292897 29350 hybrid_clock.cc:648] HybridClock initialized: now 1786388001292851 us; error 38 us; skew 500 ppm
I20260810 18:53:21.293174 29350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:21 dist-test-slave-6zbq krb5kdc[28702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:21.296066 29350 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:53:21.297308 29350 webserver.cc:460] Webserver started at http://127.29.174.195:34981/ using document root <none> and password file <none>
I20260810 18:53:21.297915 29350 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:21.298022 29350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:21.298300 29350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:21.300966 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "cba0113c18b345f29f1e235f1bad3154"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-6zbq"
server_key: "f0f977d8573eb60b46dc712d7f46a12b"
server_key_iv: "eea3a91bc0aad58ee060fc4d527e8c54"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.301712 29350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "cba0113c18b345f29f1e235f1bad3154"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-6zbq"
server_key: "f0f977d8573eb60b46dc712d7f46a12b"
server_key_iv: "eea3a91bc0aad58ee060fc4d527e8c54"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.306468 29350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 18:53:21.309527 29375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.310770 29350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:21.310914 29350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "cba0113c18b345f29f1e235f1bad3154"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-6zbq"
server_key: "f0f977d8573eb60b46dc712d7f46a12b"
server_key_iv: "eea3a91bc0aad58ee060fc4d527e8c54"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.311084 29350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:21.330642 29350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:21.334930 29350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:21.335182 29350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:21.336124 29350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:21.337273 29350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:21.337344 29350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.337425 29350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:21.337466 29350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.364383 29350 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:46573
I20260810 18:53:21.364440 29578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:46573 every 8 connection(s)
I20260810 18:53:21.365476 29350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:21.370669 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 29350
I20260810 18:53:21.370890 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:21.371444 30395 external_mini_cluster.cc:1428] Setting key dad35df27d149c216cf65b07556c8b01
Aug 10 18:53:21 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:21.386178 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.369637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:51573 (local address 127.29.174.254:43333)
0810 18:53:21.369855 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.369861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.371192 (+ 1331us) server_negotiation.cc:407] Connection header received
0810 18:53:21.373487 (+ 2295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.373493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.373573 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.373657 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.375241 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.376058 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.377533 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.377936 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.378043 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.381033 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.381038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.381041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.381083 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.383336 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.383974 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.383978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.383980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.384102 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.384514 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.384517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.384520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.384865 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.385019 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.385637 (+ 618us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.385862 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 18:53:21.386909 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.370105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:43333 (local address 127.29.174.195:51573)
0810 18:53:21.371032 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.371085 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.373157 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.373885 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.373893 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.374530 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:21.375085 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.375102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.376200 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.376203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.377401 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.377413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.378142 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.378150 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.378406 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.378926 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:21.378950 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:21.380850 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:21.383458 (+ 2608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.383466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.383479 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.383856 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.384201 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.384206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.384210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.384382 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.384998 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:21.385006 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:21.385284 (+ 278us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.385490 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.385910 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":297,"threads_started":1}
I20260810 18:53:21.388005 29581 heartbeater.cc:347] Connected to a master server at 127.29.174.254:43333
I20260810 18:53:21.388247 29581 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:21.388813 29581 heartbeater.cc:511] Master 127.29.174.254:43333 requested a full tablet report, sending...
I20260810 18:53:21.389978 28768 ts_manager.cc:194] Registered new tserver with Master: cba0113c18b345f29f1e235f1bad3154 (127.29.174.195:46573)
I20260810 18:53:21.390763 28768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:51573
I20260810 18:53:21.397953 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:21.412633 29107 heartbeater.cc:503] Master 127.29.174.254:43333 was elected leader, sending a full tablet report...
I20260810 18:53:21.412971 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.401844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.29.174.254:43333)
0810 18:53:21.401971 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.401975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.402495 (+ 520us) server_negotiation.cc:407] Connection header received
0810 18:53:21.402609 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.402612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.402664 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.402742 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.404081 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.404746 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.405864 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.406231 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.406311 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.408593 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.408598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.408601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.408626 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.410738 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.411348 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.411352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.411354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.411459 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.411777 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.411780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.411782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.412026 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.412209 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.412478 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.412640 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:53:21.416637 28767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:21.423266 28767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:21.445669 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.434812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.29.174.193:35031)
0810 18:53:21.435559 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.435567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.436284 (+ 717us) server_negotiation.cc:407] Connection header received
0810 18:53:21.436672 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.436681 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.436923 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.437075 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:21.438155 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.439790 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.443471 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.444523 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.444681 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.444802 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.445173 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.445355 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":381,"threads_started":1}
I20260810 18:53:21.445664 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.434319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35031 (local address 127.0.0.1:49014)
0810 18:53:21.436146 (+ 1827us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.436181 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.436442 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.437386 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.437392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.437421 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:21.437941 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.437958 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.439966 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.439971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.443310 (+ 3339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.443325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.444733 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.444739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.444917 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.444967 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.445098 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.445261 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1729,"mutex_wait_us":192,"spinlock_wait_cycles":3840,"thread_start_us":1411,"threads_started":1}
I20260810 18:53:21.447427 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.434310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.29.174.194:43353)
0810 18:53:21.435052 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.435060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.436286 (+ 1226us) server_negotiation.cc:407] Connection header received
0810 18:53:21.436673 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.436681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.436912 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.437075 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:21.438240 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.439798 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.444870 (+ 5072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.446166 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.446319 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.446496 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.446979 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.447146 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":660,"threads_started":1}
I20260810 18:53:21.448168 29595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.434229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43353 (local address 127.0.0.1:57316)
0810 18:53:21.436126 (+ 1897us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.436181 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.436442 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.437549 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.437554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.437578 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:21.438045 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.438056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.439966 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.439971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.443310 (+ 3339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.443325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.446563 (+ 3238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.446569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.446772 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.446805 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.447622 (+ 817us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.447762 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"mutex_wait_us":140,"spinlock_wait_cycles":6912,"thread_start_us":1496,"threads_started":1}
I20260810 18:53:21.448601 29602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.436495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:46573 (local address 127.0.0.1:51320)
0810 18:53:21.440994 (+ 4499us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.441011 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.441145 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.442039 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.442043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.442067 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:21.442493 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.442502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.443837 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.443841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.445360 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.445375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.447961 (+ 2586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.447966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.448085 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.448109 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.448191 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.448309 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4357,"thread_start_us":4186,"threads_started":1}
I20260810 18:53:21.448858 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.436848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.29.174.195:46573)
0810 18:53:21.440405 (+ 3557us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.440411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.441118 (+ 707us) server_negotiation.cc:407] Connection header received
0810 18:53:21.441345 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.441353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.441568 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.441759 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:21.442656 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.443726 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.445540 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.447566 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.447744 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.447846 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.448392 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.448547 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3428,"thread_start_us":3288,"threads_started":1}
I20260810 18:53:21.449380 28992 tablet_service.cc:1467] Processing CreateTablet for tablet a691cc0fa61c47a6abf537a3e63826ef (DEFAULT_TABLE table=test-table [id=38da8c914d904a21b67204799d3b746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:21.450590 28992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a691cc0fa61c47a6abf537a3e63826ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:21.452037 29453 tablet_service.cc:1467] Processing CreateTablet for tablet a691cc0fa61c47a6abf537a3e63826ef (DEFAULT_TABLE table=test-table [id=38da8c914d904a21b67204799d3b746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:21.453094 29453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a691cc0fa61c47a6abf537a3e63826ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:21.453531 29223 tablet_service.cc:1467] Processing CreateTablet for tablet a691cc0fa61c47a6abf537a3e63826ef (DEFAULT_TABLE table=test-table [id=38da8c914d904a21b67204799d3b746a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:21.454886 29223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a691cc0fa61c47a6abf537a3e63826ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:21.463559 29607 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Bootstrap starting.
I20260810 18:53:21.466986 29608 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Bootstrap starting.
I20260810 18:53:21.468089 29607 tablet_bootstrap.cc:654] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:21.469306 29609 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Bootstrap starting.
I20260810 18:53:21.469393 29607 log.cc:824] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:21.470525 29608 tablet_bootstrap.cc:654] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:21.471696 29608 log.cc:824] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:21.472029 29609 tablet_bootstrap.cc:654] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:21.472709 29607 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: No bootstrap required, opened a new log
I20260810 18:53:21.472957 29609 log.cc:824] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:21.473690 29607 ts_tablet_manager.cc:1397] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:53:21.475646 29609 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: No bootstrap required, opened a new log
I20260810 18:53:21.475922 29609 ts_tablet_manager.cc:1397] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:53:21.476678 29608 tablet_bootstrap.cc:492] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: No bootstrap required, opened a new log
I20260810 18:53:21.476922 29608 ts_tablet_manager.cc:1397] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260810 18:53:21.482753 29609 raft_consensus.cc:348] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.483089 29609 raft_consensus.cc:374] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:21.483180 29609 raft_consensus.cc:729] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a56d27af8f433ea779c70ff968845f, State: Initialized, Role: FOLLOWER
I20260810 18:53:21.484102 29609 consensus_queue.cc:260] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.484479 29607 raft_consensus.cc:348] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.484870 29607 raft_consensus.cc:374] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:21.484964 29607 raft_consensus.cc:729] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba0113c18b345f29f1e235f1bad3154, State: Initialized, Role: FOLLOWER
I20260810 18:53:21.485350 29343 heartbeater.cc:503] Master 127.29.174.254:43333 was elected leader, sending a full tablet report...
I20260810 18:53:21.485603 29607 consensus_queue.cc:260] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.485863 29609 ts_tablet_manager.cc:1428] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260810 18:53:21.487291 29581 heartbeater.cc:503] Master 127.29.174.254:43333 was elected leader, sending a full tablet report...
I20260810 18:53:21.487991 29607 ts_tablet_manager.cc:1428] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Time spent starting tablet: real 0.014s user 0.006s sys 0.006s
I20260810 18:53:21.486881 29608 raft_consensus.cc:348] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.488451 29608 raft_consensus.cc:374] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:21.488554 29608 raft_consensus.cc:729] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd57474b4ed24044ad292449a1ba71bf, State: Initialized, Role: FOLLOWER
I20260810 18:53:21.489100 29608 consensus_queue.cc:260] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.490681 29616 raft_consensus.cc:482] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:21.490939 29616 raft_consensus.cc:504] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.492823 29616 leader_election.cc:288] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83a56d27af8f433ea779c70ff968845f (127.29.174.194:43353), fd57474b4ed24044ad292449a1ba71bf (127.29.174.193:35031)
I20260810 18:53:21.494129 29608 ts_tablet_manager.cc:1428] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Time spent starting tablet: real 0.017s user 0.007s sys 0.007s
I20260810 18:53:21.499259 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.493678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:50659 (local address 127.29.174.194:43353)
0810 18:53:21.493870 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.493876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.493895 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:21.493959 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.493965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.494033 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.494115 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:21.494755 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.495526 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.497588 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.498409 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.498527 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.498581 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.498936 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.499022 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:21.499310 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.493198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:43353 (local address 127.29.174.195:50659)
0810 18:53:21.493404 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.493424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.493529 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.494304 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.494309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.494334 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:21.494581 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.494599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.495628 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.495633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.497399 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.497416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.498571 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.498577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.498783 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.498818 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.498928 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.499048 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:53:21.500250 29265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a691cc0fa61c47a6abf537a3e63826ef" candidate_uuid: "cba0113c18b345f29f1e235f1bad3154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a56d27af8f433ea779c70ff968845f" is_pre_election: true
I20260810 18:53:21.500728 29265 raft_consensus.cc:2454] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cba0113c18b345f29f1e235f1bad3154 in term 0.
I20260810 18:53:21.501020 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.493456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:38367 (local address 127.29.174.193:35031)
0810 18:53:21.493635 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.493641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.495393 (+ 1752us) server_negotiation.cc:407] Connection header received
0810 18:53:21.495556 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.495561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.495631 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.495713 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:21.496321 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.497268 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.499083 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.500102 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.500195 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.500250 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.500680 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.500804 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:21.501116 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.493251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35031 (local address 127.29.174.195:38367)
0810 18:53:21.495302 (+ 2051us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.495319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.495433 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.495737 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.495741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.495765 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:21.496111 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.496120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.497458 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.497462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.498928 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.498941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.500380 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.500384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.500495 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.500524 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.500638 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.500750 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1929,"mutex_wait_us":398,"thread_start_us":872,"threads_started":1}
I20260810 18:53:21.501447 29379 leader_election.cc:302] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83a56d27af8f433ea779c70ff968845f, cba0113c18b345f29f1e235f1bad3154; no voters:
I20260810 18:53:21.501842 29616 raft_consensus.cc:2781] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:21.501984 29616 raft_consensus.cc:482] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:21.502086 29616 raft_consensus.cc:3037] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:21.501951 29035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a691cc0fa61c47a6abf537a3e63826ef" candidate_uuid: "cba0113c18b345f29f1e235f1bad3154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd57474b4ed24044ad292449a1ba71bf" is_pre_election: true
I20260810 18:53:21.502465 29035 raft_consensus.cc:2454] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cba0113c18b345f29f1e235f1bad3154 in term 0.
I20260810 18:53:21.504539 29616 raft_consensus.cc:504] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.505017 29616 leader_election.cc:288] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [CANDIDATE]: Term 1 election: Requested vote from peers 83a56d27af8f433ea779c70ff968845f (127.29.174.194:43353), fd57474b4ed24044ad292449a1ba71bf (127.29.174.193:35031)
I20260810 18:53:21.505369 29265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a691cc0fa61c47a6abf537a3e63826ef" candidate_uuid: "cba0113c18b345f29f1e235f1bad3154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a56d27af8f433ea779c70ff968845f"
I20260810 18:53:21.505438 29035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a691cc0fa61c47a6abf537a3e63826ef" candidate_uuid: "cba0113c18b345f29f1e235f1bad3154" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd57474b4ed24044ad292449a1ba71bf"
I20260810 18:53:21.505527 29265 raft_consensus.cc:3037] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:21.505604 29035 raft_consensus.cc:3037] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:21.507711 29035 raft_consensus.cc:2454] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cba0113c18b345f29f1e235f1bad3154 in term 1.
I20260810 18:53:21.507736 29265 raft_consensus.cc:2454] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cba0113c18b345f29f1e235f1bad3154 in term 1.
I20260810 18:53:21.508039 29381 leader_election.cc:302] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cba0113c18b345f29f1e235f1bad3154, fd57474b4ed24044ad292449a1ba71bf; no voters:
I20260810 18:53:21.508319 29616 raft_consensus.cc:2781] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:21.508689 29616 raft_consensus.cc:686] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 1 LEADER]: Becoming Leader. State: Replica: cba0113c18b345f29f1e235f1bad3154, State: Running, Role: LEADER
I20260810 18:53:21.509177 29616 consensus_queue.cc:237] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } }
I20260810 18:53:21.513141 28767 catalog_manager.cc:5495] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 reported cstate change: term changed from 0 to 1, leader changed from <none> to cba0113c18b345f29f1e235f1bad3154 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "cba0113c18b345f29f1e235f1bad3154" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cba0113c18b345f29f1e235f1bad3154" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 46573 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:21.543028 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.538160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.29.174.195:46573)
0810 18:53:21.538374 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.538380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.538482 (+ 102us) server_negotiation.cc:407] Connection header received
0810 18:53:21.538635 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.538639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.538719 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.538830 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:21.539123 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.539815 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.541036 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.541420 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.541502 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.541963 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:21.542447 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:21.542570 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.542741 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.542778 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:21.551297 29035 raft_consensus.cc:1261] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 1 FOLLOWER]: Refusing update from remote peer cba0113c18b345f29f1e235f1bad3154: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:21.551303 29265 raft_consensus.cc:1261] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 1 FOLLOWER]: Refusing update from remote peer cba0113c18b345f29f1e235f1bad3154: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:21.552173 29619 consensus_queue.cc:1035] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83a56d27af8f433ea779c70ff968845f" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 43353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:21.552531 29616 consensus_queue.cc:1035] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd57474b4ed24044ad292449a1ba71bf" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:21.566210 29626 mvcc.cc:205] Tried to move back new op lower bound from 7317045254344462336 to 7317045254185197568. Current Snapshot: MvccSnapshot[applied={T|T < 7317045254344462336}]
I20260810 18:53:21.567745 29623 mvcc.cc:205] Tried to move back new op lower bound from 7317045254344462336 to 7317045254185197568. Current Snapshot: MvccSnapshot[applied={T|T < 7317045254344462336 or (T in {7317045254344462336})}]
I20260810 18:53:21.577399 28768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:21.577667 28768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:21.581513 28768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e6f72445370407a85b9d51cb781e2a9: Sending DeleteTablet for 3 replicas of tablet a691cc0fa61c47a6abf537a3e63826ef
I20260810 18:53:21.582597 29223 tablet_service.cc:1514] Processing DeleteTablet for tablet a691cc0fa61c47a6abf537a3e63826ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:21 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:57316
I20260810 18:53:21.582796 29453 tablet_service.cc:1514] Processing DeleteTablet for tablet a691cc0fa61c47a6abf537a3e63826ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:21 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:51320
I20260810 18:53:21.582971 28992 tablet_service.cc:1514] Processing DeleteTablet for tablet a691cc0fa61c47a6abf537a3e63826ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:21 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:49014
I20260810 18:53:21.583515 29640 tablet_replica.cc:330] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: stopping tablet replica
I20260810 18:53:21.583905 29640 raft_consensus.cc:2229] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:21.584038 29643 tablet_replica.cc:330] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: stopping tablet replica
I20260810 18:53:21.584370 29640 raft_consensus.cc:2258] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:21.584355 29643 raft_consensus.cc:2229] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:21.584769 29643 raft_consensus.cc:2258] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:21.586575 29640 ts_tablet_manager.cc:1905] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:21.586900 29643 ts_tablet_manager.cc:1905] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:21.587287 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 28891
I20260810 18:53:21.593539 29640 ts_tablet_manager.cc:1918] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:21.593689 29640 log.cc:1196] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/a691cc0fa61c47a6abf537a3e63826ef
I20260810 18:53:21.593847 29643 ts_tablet_manager.cc:1918] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:21.594018 29643 log.cc:1196] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/a691cc0fa61c47a6abf537a3e63826ef
I20260810 18:53:21.594226 29640 ts_tablet_manager.cc:1939] T a691cc0fa61c47a6abf537a3e63826ef P 83a56d27af8f433ea779c70ff968845f: Deleting consensus metadata
I20260810 18:53:21.594558 29643 ts_tablet_manager.cc:1939] T a691cc0fa61c47a6abf537a3e63826ef P fd57474b4ed24044ad292449a1ba71bf: Deleting consensus metadata
I20260810 18:53:21.597214 28750 catalog_manager.cc:4841] TS 83a56d27af8f433ea779c70ff968845f (127.29.174.194:43353): tablet a691cc0fa61c47a6abf537a3e63826ef (table test-table [id=38da8c914d904a21b67204799d3b746a]) successfully deleted
I20260810 18:53:21.597856 28753 catalog_manager.cc:4841] TS fd57474b4ed24044ad292449a1ba71bf (127.29.174.193:35031): tablet a691cc0fa61c47a6abf537a3e63826ef (table test-table [id=38da8c914d904a21b67204799d3b746a]) successfully deleted
I20260810 18:53:21.598814 29642 tablet_replica.cc:330] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: stopping tablet replica
I20260810 18:53:21.599186 29642 raft_consensus.cc:2229] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:21.599732 29642 raft_consensus.cc:2258] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:21.601938 29642 ts_tablet_manager.cc:1905] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:21.607988 29642 ts_tablet_manager.cc:1918] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:21.608179 29642 log.cc:1196] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/a691cc0fa61c47a6abf537a3e63826ef
I20260810 18:53:21.608634 29642 ts_tablet_manager.cc:1939] T a691cc0fa61c47a6abf537a3e63826ef P cba0113c18b345f29f1e235f1bad3154: Deleting consensus metadata
Aug 10 18:53:21 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:53:21.610201 28750 catalog_manager.cc:4841] TS cba0113c18b345f29f1e235f1bad3154 (127.29.174.195:46573): tablet a691cc0fa61c47a6abf537a3e63826ef (table test-table [id=38da8c914d904a21b67204799d3b746a]) successfully deleted
I20260810 18:53:21.616884 29600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.588454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.29.174.193:35031)
0810 18:53:21.588625 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.588631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.598152 (+ 9521us) server_negotiation.cc:407] Connection header received
0810 18:53:21.598379 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.598384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.598474 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.598573 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.600369 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.601476 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.603522 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.604128 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.604228 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.610059 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.610078 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.610097 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.610153 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.613758 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.614557 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.614568 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.614572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.614741 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.615094 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.615100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.615103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.615566 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.615877 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.616445 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.616613 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:21.617576 29096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:21.704401 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 28891
I20260810 18:53:21.732846 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 29118
Aug 10 18:53:21 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:53:21.751889 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.733764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.29.174.194:43353)
0810 18:53:21.733942 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.733948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.733969 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:21.734031 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.734036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.734110 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.734229 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.735909 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.737013 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.738994 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.739625 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.739747 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.745130 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.745150 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.745172 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.745216 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.748959 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.749622 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.749632 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.749636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.749791 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.750109 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.750114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.750118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.750822 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.751134 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.751323 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.751531 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:53:21.752557 29334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:21.839781 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 29118
I20260810 18:53:21.877462 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 29350
Aug 10 18:53:21 dist-test-slave-6zbq krb5kdc[28702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:53:21.894101 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.878184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.29.174.195:46573)
0810 18:53:21.878408 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.878414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.878436 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:53:21.878559 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.878566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.878660 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.878791 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.880041 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.881270 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.882755 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.883415 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.883546 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.888287 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.888301 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.888317 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.888349 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.891332 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.892150 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.892156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.892159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.892318 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.892712 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.892715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.892717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.893063 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.893318 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.893657 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.893810 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:53:21.894872 29566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:22.014118 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 29350
I20260810 18:53:22.038663 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 28720
I20260810 18:53:22.039803 28822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:22.145768 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 28720
2026-08-10T18:53:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4327 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T18:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:22Z Disabled control of system clock
I20260810 18:53:22.230990 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41061
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:41061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:22.552986 29662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:22.553267 29662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:22.553347 29662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:22.561187 29662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:22.561301 29662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:22.561347 29662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:22.561409 29662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:22.561472 29662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:22.570683 29662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:41061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:41061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:22.571918 29662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:22.573719 29662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:22.584784 29676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:22.584748 29678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:22.584731 29680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:22.585695 29662 server_base.cc:1034] running on GCE node
I20260810 18:53:22.586428 29662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:22.587651 29662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:22.588842 29662 hybrid_clock.cc:648] HybridClock initialized: now 1786388002588805 us; error 31 us; skew 500 ppm
I20260810 18:53:22.589126 29662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:22.591207 29662 webserver.cc:460] Webserver started at http://127.29.174.254:38781/ using document root <none> and password file <none>
I20260810 18:53:22.591786 29662 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:22.591869 29662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:22.592118 29662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:22.594806 29662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-6zbq"
server_key: "1834bfae0db47ab86d8a6874fe114d7a"
server_key_iv: "50ce5fcdb8921023a62e301baf105e50"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.595523 29662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-6zbq"
server_key: "1834bfae0db47ab86d8a6874fe114d7a"
server_key_iv: "50ce5fcdb8921023a62e301baf105e50"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.600106 29662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:22.603183 29688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:22.604741 29662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:22.604911 29662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-6zbq"
server_key: "1834bfae0db47ab86d8a6874fe114d7a"
server_key_iv: "50ce5fcdb8921023a62e301baf105e50"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.605139 29662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:22.633692 29662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:22.634449 29662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:22.634681 29662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:22.652087 29662 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:41061
I20260810 18:53:22.652129 29783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:41061 every 8 connection(s)
I20260810 18:53:22.653318 29662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:22.656883 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.661988 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 29662
I20260810 18:53:22.662138 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:22.662611 30395 external_mini_cluster.cc:1428] Setting key 321e9584279e509247a0425ed43b6750
I20260810 18:53:22.666973 29784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: Bootstrap starting.
I20260810 18:53:22.670073 29784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.671291 29784 log.cc:824] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:22.674137 29784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: No bootstrap required, opened a new log
I20260810 18:53:22.676074 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.664403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.29.174.254:41061)
0810 18:53:22.665386 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.665396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.665438 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:53:22.667065 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.667088 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.667503 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.667786 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.668369 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.669627 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.671530 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.672275 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.672406 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.673450 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.673461 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.673478 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:22.674139 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.674248 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:22.674273 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.674422 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.674600 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.674902 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":379,"threads_started":1}
I20260810 18:53:22.681731 29784 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } }
I20260810 18:53:22.682041 29784 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.682113 29784 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14c8fda3cfb4f11a7e7086d8ca28d01, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.682600 29784 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } }
I20260810 18:53:22.682809 29784 raft_consensus.cc:388] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:22.682925 29784 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:22.683048 29784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.685321 29784 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } }
I20260810 18:53:22.685736 29784 leader_election.cc:302] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f14c8fda3cfb4f11a7e7086d8ca28d01; no voters:
I20260810 18:53:22.686199 29784 leader_election.cc:288] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:22.686333 29792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:22.686997 29792 raft_consensus.cc:686] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [term 1 LEADER]: Becoming Leader. State: Replica: f14c8fda3cfb4f11a7e7086d8ca28d01, State: Running, Role: LEADER
I20260810 18:53:22.687465 29792 consensus_queue.cc:237] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } }
I20260810 18:53:22.688050 29784 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:22.690181 29794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } } }
I20260810 18:53:22.690232 29796 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f14c8fda3cfb4f11a7e7086d8ca28d01. Latest consensus state: current_term: 1 leader_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14c8fda3cfb4f11a7e7086d8ca28d01" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 41061 } } }
I20260810 18:53:22.690635 29796 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:22.690640 29794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:22.695396 29802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:22.699770 29802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:22.708189 29802 catalog_manager.cc:1347] Generated new cluster ID: 694832ad9d2147deb7e52c2daf669811
I20260810 18:53:22.708285 29802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:22.743394 29802 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:22.744772 29802 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:22.752161 29802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: Generated new TSK 0
I20260810 18:53:22.753005 29802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:22.759862 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--builtin_ntp_servers=127.29.174.212:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:23.079816 29819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:23.080061 29819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:23.080152 29819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:23.087157 29819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:23.087256 29819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:23.087361 29819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:23.096339 29819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:23.097568 29819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:23.099262 29819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:23.111234 29830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.111276 29831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.111455 29833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:23.112258 29819 server_base.cc:1034] running on GCE node
I20260810 18:53:23.112775 29819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:23.113581 29819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:23.114775 29819 hybrid_clock.cc:648] HybridClock initialized: now 1786388003114734 us; error 33 us; skew 500 ppm
I20260810 18:53:23.115059 29819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:23.117049 29819 webserver.cc:460] Webserver started at http://127.29.174.193:38751/ using document root <none> and password file <none>
I20260810 18:53:23.117669 29819 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:23.117758 29819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:23.118034 29819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:23.121044 29819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "f62063c6edecaeb84e6600d1e3a190fd"
server_key_iv: "fc4f8cf698b72edbaeb4466c06d118ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.121840 29819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "f62063c6edecaeb84e6600d1e3a190fd"
server_key_iv: "fc4f8cf698b72edbaeb4466c06d118ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.126549 29819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 18:53:23.129637 29848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.130860 29819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:23.131003 29819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "f62063c6edecaeb84e6600d1e3a190fd"
server_key_iv: "fc4f8cf698b72edbaeb4466c06d118ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.131187 29819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:23.155843 29819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:23.156711 29819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:23.156991 29819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:23.157927 29819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:23.159375 29819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:23.159472 29819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.159580 29819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:23.159654 29819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.189285 29819 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:46551
I20260810 18:53:23.189410 30051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:46551 every 8 connection(s)
I20260810 18:53:23.190554 29819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:23.197443 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 29819
I20260810 18:53:23.197583 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:23.197999 30395 external_mini_cluster.cc:1428] Setting key dc0a49ecc7c68492644c2afbc98bbad7
I20260810 18:53:23.200850 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--builtin_ntp_servers=127.29.174.212:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:23.202701 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.192934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:50463 (local address 127.29.174.254:41061)
0810 18:53:23.193328 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.193333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.194371 (+ 1038us) server_negotiation.cc:407] Connection header received
0810 18:53:23.196120 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.196126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.196203 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.196301 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:23.197328 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.198130 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.199846 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.200419 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.200556 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.201817 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:23.201824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:23.201829 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:23.201850 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:23.201898 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:23.201915 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:23.202001 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.202195 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.202312 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":206,"threads_started":1}
I20260810 18:53:23.203544 30060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.193367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41061 (local address 127.29.174.193:50463)
0810 18:53:23.194204 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.194243 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.195869 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.196530 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.196537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.196623 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:23.197169 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.197187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.198280 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.198284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.199704 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.199720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.200613 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.200618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.200823 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.201518 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:23.201521 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:23.201556 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:23.201632 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:23.202009 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:23.202019 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:23.202027 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.202168 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.202510 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":245,"threads_started":1}
I20260810 18:53:23.204878 30054 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41061
I20260810 18:53:23.205168 30054 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:23.205952 30054 heartbeater.cc:511] Master 127.29.174.254:41061 requested a full tablet report, sending...
I20260810 18:53:23.207918 29715 ts_manager.cc:194] Registered new tserver with Master: e12d8b99a52d4a6fb6aeb1ba28d10a9f (127.29.174.193:46551)
I20260810 18:53:23.209368 29715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.193:50463
W20260810 18:53:23.522102 30062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:23.522361 30062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:23.522477 30062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:23.529285 30062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:23.529377 30062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:23.529520 30062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:23.538381 30062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:23.539386 30062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:23.540949 30062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:23.552567 30069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.552619 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.552560 30074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:23.553457 30062 server_base.cc:1034] running on GCE node
I20260810 18:53:23.553979 30062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:23.554844 30062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:23.556030 30062 hybrid_clock.cc:648] HybridClock initialized: now 1786388003555991 us; error 33 us; skew 500 ppm
I20260810 18:53:23.556300 30062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:23.558375 30062 webserver.cc:460] Webserver started at http://127.29.174.194:38593/ using document root <none> and password file <none>
I20260810 18:53:23.558992 30062 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:23.559077 30062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:23.559356 30062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:23.561944 30062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "308758e351824ae2b338869ae2a55405"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "709d71244b205bf379f9d09135c39599"
server_key_iv: "a57983048e3cda5dbfb62e0d419820da"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.562701 30062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "308758e351824ae2b338869ae2a55405"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "709d71244b205bf379f9d09135c39599"
server_key_iv: "a57983048e3cda5dbfb62e0d419820da"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.567242 30062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:53:23.570292 30087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.571481 30062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:23.571640 30062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "308758e351824ae2b338869ae2a55405"
format_stamp: "Formatted at 2026-08-10 18:53:23 on dist-test-slave-6zbq"
server_key: "709d71244b205bf379f9d09135c39599"
server_key_iv: "a57983048e3cda5dbfb62e0d419820da"
server_key_version: "encryptionkey@0"
I20260810 18:53:23.571825 30062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:23.607272 30062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:23.607975 30062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:23.608232 30062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:23.609076 30062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:23.610545 30062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:23.610642 30062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.610740 30062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:23.610810 30062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:23.640450 30062 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:35407
I20260810 18:53:23.640508 30293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:35407 every 8 connection(s)
I20260810 18:53:23.641644 30062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:23.649236 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30062
I20260810 18:53:23.649398 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:23.649789 30395 external_mini_cluster.cc:1428] Setting key 5ab75b0e610a71d953d3fabb1fe9bfb3
I20260810 18:53:23.652745 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--builtin_ntp_servers=127.29.174.212:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:23.653282 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.643886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:47513 (local address 127.29.174.254:41061)
0810 18:53:23.644061 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.644066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.645342 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 18:53:23.647086 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.647093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.647184 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.647273 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:23.648290 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.648987 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.650728 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.651154 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.651268 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.652535 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:23.652540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:23.652544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:23.652560 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:23.652591 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:23.652603 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:23.652680 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.652920 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.653014 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:23.654680 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.644341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41061 (local address 127.29.174.194:47513)
0810 18:53:23.645182 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.645227 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.646828 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.647510 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.647517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.647595 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:23.648119 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.648134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.649155 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.649159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.650580 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.650593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.651385 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.651394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.651608 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.652245 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:23.652248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:23.652283 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:23.652356 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:23.652695 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:23.652707 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:23.652715 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.653469 (+ 754us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.653758 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":250,"threads_started":1}
I20260810 18:53:23.658428 30295 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41061
I20260810 18:53:23.658699 30295 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:23.659350 30295 heartbeater.cc:511] Master 127.29.174.254:41061 requested a full tablet report, sending...
I20260810 18:53:23.660602 29715 ts_manager.cc:194] Registered new tserver with Master: 308758e351824ae2b338869ae2a55405 (127.29.174.194:35407)
I20260810 18:53:23.661397 29715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.194:47513
W20260810 18:53:23.967475 30299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:23.967723 30299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:23.967839 30299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:23.974767 30299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:23.974862 30299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:23.974999 30299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:23.983601 30299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:41061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:23.984720 30299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:23.986392 30299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:23.998266 30322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.998242 30317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:23.998579 30316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:23.998998 30299 server_base.cc:1034] running on GCE node
I20260810 18:53:23.999495 30299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:24.000280 30299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:24.001462 30299 hybrid_clock.cc:648] HybridClock initialized: now 1786388004001419 us; error 36 us; skew 500 ppm
I20260810 18:53:24.001724 30299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:24.003839 30299 webserver.cc:460] Webserver started at http://127.29.174.195:42415/ using document root <none> and password file <none>
I20260810 18:53:24.004413 30299 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:24.004496 30299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:24.004750 30299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:24.007476 30299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "d3aae1af9fec4d2fa9354637d684b553"
format_stamp: "Formatted at 2026-08-10 18:53:24 on dist-test-slave-6zbq"
server_key: "fb7fa5956c7cf8c72e4ce1c207965c52"
server_key_iv: "70454f64e28a818486d73217a7839052"
server_key_version: "encryptionkey@0"
I20260810 18:53:24.008193 30299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "d3aae1af9fec4d2fa9354637d684b553"
format_stamp: "Formatted at 2026-08-10 18:53:24 on dist-test-slave-6zbq"
server_key: "fb7fa5956c7cf8c72e4ce1c207965c52"
server_key_iv: "70454f64e28a818486d73217a7839052"
server_key_version: "encryptionkey@0"
I20260810 18:53:24.012887 30299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 18:53:24.016000 30329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:24.017131 30299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:24.017251 30299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "d3aae1af9fec4d2fa9354637d684b553"
format_stamp: "Formatted at 2026-08-10 18:53:24 on dist-test-slave-6zbq"
server_key: "fb7fa5956c7cf8c72e4ce1c207965c52"
server_key_iv: "70454f64e28a818486d73217a7839052"
server_key_version: "encryptionkey@0"
I20260810 18:53:24.017416 30299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:24.029023 30299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:24.029678 30299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:24.029918 30299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:24.030783 30299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:24.032063 30299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:24.032145 30299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:24.032222 30299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:24.032274 30299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:24.061609 30299 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:42145
I20260810 18:53:24.061671 30533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:42145 every 8 connection(s)
I20260810 18:53:24.062815 30299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:24.070915 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30299
I20260810 18:53:24.071065 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:24.071498 30395 external_mini_cluster.cc:1428] Setting key d1558fbf4656d2ed0466cbe82dbc7678
I20260810 18:53:24.074025 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.065044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:54487 (local address 127.29.174.254:41061)
0810 18:53:24.065260 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.065266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.066453 (+ 1187us) server_negotiation.cc:407] Connection header received
0810 18:53:24.068068 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.068072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.068137 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.068219 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.069118 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.069879 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.071466 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.071914 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.072028 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.073296 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.073301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.073304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:24.073320 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.073354 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:24.073365 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.073430 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.073640 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.073719 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:53:24.075057 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.065494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:41061 (local address 127.29.174.195:54487)
0810 18:53:24.066303 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.066364 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.067861 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.068382 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.068389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.068473 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:24.068997 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.069012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.070022 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.070026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.071329 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.071342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.072122 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.072130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.072394 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.073003 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:24.073006 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:24.073046 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:24.073134 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:24.073451 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:24.073460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:24.073468 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.073655 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.074012 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":249,"threads_started":1}
I20260810 18:53:24.076277 30536 heartbeater.cc:347] Connected to a master server at 127.29.174.254:41061
I20260810 18:53:24.076524 30536 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:24.077152 30536 heartbeater.cc:511] Master 127.29.174.254:41061 requested a full tablet report, sending...
I20260810 18:53:24.078423 29715 ts_manager.cc:194] Registered new tserver with Master: d3aae1af9fec4d2fa9354637d684b553 (127.29.174.195:42145)
I20260810 18:53:24.079152 29715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.195:54487
I20260810 18:53:24.086320 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:24.100291 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.092881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.29.174.254:41061)
0810 18:53:24.093376 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.093382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.093682 (+ 300us) server_negotiation.cc:407] Connection header received
0810 18:53:24.093740 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.093745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.093833 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.093925 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.094677 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.095685 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.097612 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.098235 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.098350 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.099536 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.099541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.099545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:24.099563 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.099596 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:24.099609 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.099736 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.099936 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.099988 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323}
I20260810 18:53:24.106067 29715 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:24.115324 29715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:24.138568 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.128013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49348)
0810 18:53:24.128962 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.129003 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.129147 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.131279 (+ 2132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.131287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.131316 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.131806 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.131820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.133668 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.133673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.135925 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.135943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.137717 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.137734 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.137929 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.137980 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.138077 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.138248 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":3590,"threads_started":1}
I20260810 18:53:24.139865 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.128216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.29.174.193:46551)
0810 18:53:24.130741 (+ 2525us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.130758 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.130788 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:53:24.130873 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.130878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.131092 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.131281 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.132031 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.133514 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.136169 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.137526 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.139066 (+ 1540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.139286 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.139356 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.139516 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2371,"thread_start_us":2268,"threads_started":1}
I20260810 18:53:24.140333 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.128972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.29.174.194:35407)
0810 18:53:24.129588 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.129593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.132005 (+ 2412us) server_negotiation.cc:407] Connection header received
0810 18:53:24.132303 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.132310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.132509 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.132652 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.133462 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.134957 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.137657 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.138999 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.139150 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.139326 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.139881 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.140035 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":370,"threads_started":1}
I20260810 18:53:24.140872 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.128328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:35407 (local address 127.0.0.1:41730)
0810 18:53:24.131913 (+ 3585us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.131929 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.132104 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.132927 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.132932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.132955 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.133293 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.133303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.135422 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.135428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.137446 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.137462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.139493 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.139499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.139683 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.139714 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.140538 (+ 824us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.140636 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3445,"mutex_wait_us":3,"thread_start_us":361,"threads_started":1}
I20260810 18:53:24.144502 30175 tablet_service.cc:1467] Processing CreateTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 (DEFAULT_TABLE table=test-table [id=22c9006a25484a8f8218df602e222cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.145680 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cee9997b9164f0d8e7fad9862e65880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.146492 29934 tablet_service.cc:1467] Processing CreateTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 (DEFAULT_TABLE table=test-table [id=22c9006a25484a8f8218df602e222cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.147687 29934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cee9997b9164f0d8e7fad9862e65880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.152837 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.132016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:42145 (local address 127.0.0.1:49502)
0810 18:53:24.138787 (+ 6771us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.138804 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.138893 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.146505 (+ 7612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.146510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.146535 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.146673 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.146683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.148698 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.148703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.150499 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.150514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.152147 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.152154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.152338 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.152370 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.152452 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.152578 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6665,"thread_start_us":11600,"threads_started":1}
I20260810 18:53:24.153038 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.132343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.29.174.195:42145)
0810 18:53:24.142798 (+ 10455us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.142806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.142837 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:53:24.142921 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.142927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.143128 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.143313 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.146847 (+ 3534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.148534 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.150664 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.151975 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.152163 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.152350 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.152529 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.152692 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10301,"thread_start_us":10729,"threads_started":1}
I20260810 18:53:24.157987 30565 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Bootstrap starting.
I20260810 18:53:24.157752 30416 tablet_service.cc:1467] Processing CreateTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 (DEFAULT_TABLE table=test-table [id=22c9006a25484a8f8218df602e222cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.159129 30416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cee9997b9164f0d8e7fad9862e65880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.161540 30565 tablet_bootstrap.cc:654] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.162815 30565 log.cc:824] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:24.165738 30567 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: Bootstrap starting.
I20260810 18:53:24.169075 30567 tablet_bootstrap.cc:654] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.169399 30565 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: No bootstrap required, opened a new log
I20260810 18:53:24.169507 30573 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Bootstrap starting.
I20260810 18:53:24.170001 30565 ts_tablet_manager.cc:1397] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20260810 18:53:24.170262 30567 log.cc:824] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:24.172544 30573 tablet_bootstrap.cc:654] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.173619 30573 log.cc:824] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:24.175176 30567 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: No bootstrap required, opened a new log
I20260810 18:53:24.175429 30567 ts_tablet_manager.cc:1397] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:53:24.176115 30573 tablet_bootstrap.cc:492] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: No bootstrap required, opened a new log
I20260810 18:53:24.176340 30573 ts_tablet_manager.cc:1397] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 18:53:24.179723 30565 raft_consensus.cc:348] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.180135 30565 raft_consensus.cc:374] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.180227 30565 raft_consensus.cc:729] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308758e351824ae2b338869ae2a55405, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.180909 30565 consensus_queue.cc:260] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.184554 30573 raft_consensus.cc:348] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.184955 30573 raft_consensus.cc:374] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.185050 30573 raft_consensus.cc:729] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3aae1af9fec4d2fa9354637d684b553, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.185595 30573 consensus_queue.cc:260] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.185673 30567 raft_consensus.cc:348] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.186059 30567 raft_consensus.cc:374] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.186170 30567 raft_consensus.cc:729] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12d8b99a52d4a6fb6aeb1ba28d10a9f, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.186722 30567 consensus_queue.cc:260] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.187851 30054 heartbeater.cc:503] Master 127.29.174.254:41061 was elected leader, sending a full tablet report...
I20260810 18:53:24.196847 30567 ts_tablet_manager.cc:1428] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: Time spent starting tablet: real 0.021s user 0.013s sys 0.000s
I20260810 18:53:24.199874 30295 heartbeater.cc:503] Master 127.29.174.254:41061 was elected leader, sending a full tablet report...
I20260810 18:53:24.200573 30565 ts_tablet_manager.cc:1428] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Time spent starting tablet: real 0.030s user 0.016s sys 0.004s
I20260810 18:53:24.201095 30536 heartbeater.cc:503] Master 127.29.174.254:41061 was elected leader, sending a full tablet report...
I20260810 18:53:24.203480 30573 ts_tablet_manager.cc:1428] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Time spent starting tablet: real 0.027s user 0.019s sys 0.005s
I20260810 18:53:24.293066 30578 raft_consensus.cc:482] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:24.293282 30578 raft_consensus.cc:504] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.294808 30578 leader_election.cc:288] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e12d8b99a52d4a6fb6aeb1ba28d10a9f (127.29.174.193:46551), 308758e351824ae2b338869ae2a55405 (127.29.174.194:35407)
I20260810 18:53:24.300832 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.295255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:35689 (local address 127.29.174.193:46551)
0810 18:53:24.295445 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.295451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.295470 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:24.295735 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.295741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.295816 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.295906 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.296326 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.297228 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.299095 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.300064 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.300175 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.300223 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.300518 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.300601 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:53:24.300927 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.295081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.29.174.195:35689)
0810 18:53:24.295375 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.295400 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.295523 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.295961 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.295964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.295989 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.296160 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.296172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.297353 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.297359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.298919 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.298945 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.300207 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.300213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.300376 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.300410 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.300504 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.300633 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 18:53:24.301718 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.295481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:53313 (local address 127.29.174.194:35407)
0810 18:53:24.295670 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.295676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.296132 (+ 456us) server_negotiation.cc:407] Connection header received
0810 18:53:24.296289 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.296294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.296362 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.296426 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.296973 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.297854 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.299725 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.300659 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.300829 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.300890 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.301416 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.301542 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:53:24.301739 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.295221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:35407 (local address 127.29.174.195:53313)
0810 18:53:24.296030 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.296046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.296154 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.296464 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.296468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.296490 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.296810 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.296822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.298011 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.298015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.299555 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.299570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.300848 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.300854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.301269 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.301293 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.301361 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.301436 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":66,"thread_start_us":387,"threads_started":1}
I20260810 18:53:24.301970 29962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cee9997b9164f0d8e7fad9862e65880" candidate_uuid: "d3aae1af9fec4d2fa9354637d684b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" is_pre_election: true
I20260810 18:53:24.302475 29962 raft_consensus.cc:2454] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3aae1af9fec4d2fa9354637d684b553 in term 0.
I20260810 18:53:24.302659 30205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cee9997b9164f0d8e7fad9862e65880" candidate_uuid: "d3aae1af9fec4d2fa9354637d684b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308758e351824ae2b338869ae2a55405" is_pre_election: true
I20260810 18:53:24.303041 30332 leader_election.cc:302] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3aae1af9fec4d2fa9354637d684b553, e12d8b99a52d4a6fb6aeb1ba28d10a9f; no voters:
I20260810 18:53:24.303064 30205 raft_consensus.cc:2454] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3aae1af9fec4d2fa9354637d684b553 in term 0.
I20260810 18:53:24.303346 30578 raft_consensus.cc:2781] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:24.303470 30578 raft_consensus.cc:482] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:24.303581 30578 raft_consensus.cc:3037] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.305672 30578 raft_consensus.cc:504] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
I20260810 18:53:24.306244 30578 leader_election.cc:288] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [CANDIDATE]: Term 1 election: Requested vote from peers e12d8b99a52d4a6fb6aeb1ba28d10a9f (127.29.174.193:46551), 308758e351824ae2b338869ae2a55405 (127.29.174.194:35407)
I20260810 18:53:24.306581 29962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cee9997b9164f0d8e7fad9862e65880" candidate_uuid: "d3aae1af9fec4d2fa9354637d684b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f"
I20260810 18:53:24.306708 30205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cee9997b9164f0d8e7fad9862e65880" candidate_uuid: "d3aae1af9fec4d2fa9354637d684b553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308758e351824ae2b338869ae2a55405"
I20260810 18:53:24.306788 29962 raft_consensus.cc:3037] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.306891 30205 raft_consensus.cc:3037] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.309935 29962 raft_consensus.cc:2454] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3aae1af9fec4d2fa9354637d684b553 in term 1.
I20260810 18:53:24.309949 30205 raft_consensus.cc:2454] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3aae1af9fec4d2fa9354637d684b553 in term 1.
I20260810 18:53:24.310317 30335 leader_election.cc:302] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308758e351824ae2b338869ae2a55405, d3aae1af9fec4d2fa9354637d684b553; no voters:
I20260810 18:53:24.310644 30578 raft_consensus.cc:2781] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:24.311064 30578 raft_consensus.cc:686] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 1 LEADER]: Becoming Leader. State: Replica: d3aae1af9fec4d2fa9354637d684b553, State: Running, Role: LEADER
I20260810 18:53:24.311486 30578 consensus_queue.cc:237] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } }
W20260810 18:53:24.314257 30540 tablet.cc:2369] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:24.315847 29714 catalog_manager.cc:5495] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3aae1af9fec4d2fa9354637d684b553 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "d3aae1af9fec4d2fa9354637d684b553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3aae1af9fec4d2fa9354637d684b553" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 42145 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:24.336040 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.331061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.29.174.195:42145)
0810 18:53:24.331230 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.331235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.331249 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:24.331314 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.331318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.331384 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.331495 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:24.331801 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.332483 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.333848 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.334299 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.334457 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.335032 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:24.335489 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:24.335653 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.335799 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.335839 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:24.343407 30205 raft_consensus.cc:1261] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 1 FOLLOWER]: Refusing update from remote peer d3aae1af9fec4d2fa9354637d684b553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:24.343475 29962 raft_consensus.cc:1261] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 1 FOLLOWER]: Refusing update from remote peer d3aae1af9fec4d2fa9354637d684b553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:24.344465 30578 consensus_queue.cc:1035] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "308758e351824ae2b338869ae2a55405" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:24.345000 30584 consensus_queue.cc:1035] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e12d8b99a52d4a6fb6aeb1ba28d10a9f" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 46551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:24.357139 30593 mvcc.cc:205] Tried to move back new op lower bound from 7317045265781506048 to 7317045265663393792. Current Snapshot: MvccSnapshot[applied={T|T < 7317045265781506048}]
I20260810 18:53:24.372098 29715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:24.372354 29715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:24.375572 29715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f14c8fda3cfb4f11a7e7086d8ca28d01: Sending DeleteTablet for 3 replicas of tablet 8cee9997b9164f0d8e7fad9862e65880
I20260810 18:53:24.376583 30175 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='slave'} at 127.0.0.1:41730
I20260810 18:53:24.376612 29934 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='slave'} at 127.0.0.1:49348
I20260810 18:53:24.376838 30416 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cee9997b9164f0d8e7fad9862e65880 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='slave'} at 127.0.0.1:49502
I20260810 18:53:24.379945 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 29819
I20260810 18:53:24.388137 30613 tablet_replica.cc:330] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: stopping tablet replica
I20260810 18:53:24.388144 30609 tablet_replica.cc:330] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: stopping tablet replica
I20260810 18:53:24.388514 30609 raft_consensus.cc:2229] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:24.388569 30613 raft_consensus.cc:2229] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:24.388990 30609 raft_consensus.cc:2258] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.389503 30613 raft_consensus.cc:2258] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.391234 30609 ts_tablet_manager.cc:1905] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:24.391438 30613 ts_tablet_manager.cc:1905] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:24.397790 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.381334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.29.174.193:46551)
0810 18:53:24.381498 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.381504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.385104 (+ 3600us) server_negotiation.cc:407] Connection header received
0810 18:53:24.385281 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.385287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.385395 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.385502 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.390588 (+ 5086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.391712 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.393592 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.394253 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.394371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.396189 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.396201 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.396216 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:24.396894 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.396964 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:24.396989 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.397074 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.397316 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.397426 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:24.398226 30613 ts_tablet_manager.cc:1918] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:24.398615 30611 tablet_replica.cc:330] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f: stopping tablet replica
I20260810 18:53:24.398752 30609 ts_tablet_manager.cc:1918] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:24.398846 30044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:24.398983 30609 log.cc:1196] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/8cee9997b9164f0d8e7fad9862e65880
I20260810 18:53:24.399191 30613 log.cc:1196] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/8cee9997b9164f0d8e7fad9862e65880
I20260810 18:53:24.399663 30609 ts_tablet_manager.cc:1939] T 8cee9997b9164f0d8e7fad9862e65880 P 308758e351824ae2b338869ae2a55405: Deleting consensus metadata
I20260810 18:53:24.399679 30613 ts_tablet_manager.cc:1939] T 8cee9997b9164f0d8e7fad9862e65880 P d3aae1af9fec4d2fa9354637d684b553: Deleting consensus metadata
I20260810 18:53:24.401283 29692 catalog_manager.cc:4841] TS 308758e351824ae2b338869ae2a55405 (127.29.174.194:35407): tablet 8cee9997b9164f0d8e7fad9862e65880 (table test-table [id=22c9006a25484a8f8218df602e222cd4]) successfully deleted
I20260810 18:53:24.401615 29690 catalog_manager.cc:4841] TS d3aae1af9fec4d2fa9354637d684b553 (127.29.174.195:42145): tablet 8cee9997b9164f0d8e7fad9862e65880 (table test-table [id=22c9006a25484a8f8218df602e222cd4]) successfully deleted
I20260810 18:53:24.399034 30611 raft_consensus.cc:2229] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:24.486730 30611 raft_consensus.cc:2258] T 8cee9997b9164f0d8e7fad9862e65880 P e12d8b99a52d4a6fb6aeb1ba28d10a9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.486842 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 29819
W20260810 18:53:24.515358 29690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:46551 (error 108)
I20260810 18:53:24.515489 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30062
I20260810 18:53:24.517503 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.516985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49368)
0810 18:53:24.517186 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.517322 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 18:53:24.517786 29690 catalog_manager.cc:4584] TS e12d8b99a52d4a6fb6aeb1ba28d10a9f (127.29.174.193:46551): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8cee9997b9164f0d8e7fad9862e65880: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
I20260810 18:53:24.522748 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.516682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.29.174.194:35407)
0810 18:53:24.516860 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.516866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.516886 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:53:24.516952 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.516957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.517070 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.517177 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.517686 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.518625 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.519823 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.520262 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.520372 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.521410 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.521419 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.521432 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:24.521987 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.522032 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:24.522052 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.522148 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.522291 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.522424 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:53:24.523425 30276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:24.571655 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.570964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49372)
0810 18:53:24.571330 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.571449 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:53:24.572876 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.572450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49382)
0810 18:53:24.572630 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.572717 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:53:24.611747 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30062
I20260810 18:53:24.633486 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.633040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49396)
0810 18:53:24.633257 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.633342 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 18:53:24.634554 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.634139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49412)
0810 18:53:24.634291 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.634401 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:53:24.640987 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30299
I20260810 18:53:24.648504 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.641719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.29.174.195:42145)
0810 18:53:24.641859 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.641864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.641881 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:24.642037 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.642041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.642133 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.642209 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.642721 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.643526 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.645395 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.645868 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.645967 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.647173 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.647182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.647197 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:24.647784 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.647835 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:24.647858 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.647939 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.648111 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.648187 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 18:53:24.649111 30524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:24.715847 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.715401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49418)
0810 18:53:24.715604 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.715699 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:53:24.716861 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.716484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49434)
0810 18:53:24.716673 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.716748 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260810 18:53:24.765816 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30299
I20260810 18:53:24.789067 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 29662
I20260810 18:53:24.790126 29777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:24.894719 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.894107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:46551 (local address 127.0.0.1:49436)
0810 18:53:24.894326 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.894468 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:46551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:53:24.894932 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 29662
2026-08-10T18:53:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2746 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:25 dist-test-slave-6zbq krb5kdc[30639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:25 dist-test-slave-6zbq krb5kdc[30639](info): set up 2 sockets
Aug 10 18:53:25 dist-test-slave-6zbq krb5kdc[30639](info): commencing operation
krb5kdc: starting...
W20260810 18:53:26.130470 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:26 dist-test-slave-6zbq krb5kdc[30639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:53:26.409237 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46047
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:46047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:26.734187 30653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:26.734452 30653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:26.734578 30653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:26.741541 30653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:26.741636 30653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:26.741688 30653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:26.741780 30653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:26.741832 30653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:26.751259 30653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:46047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:46047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:26.752496 30653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:26.753713 30653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:26.764921 30660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:26.764954 30672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:26.764881 30664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:26.765655 30653 server_base.cc:1034] running on GCE node
I20260810 18:53:26.766399 30653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:26.767652 30653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:26.768848 30653 hybrid_clock.cc:648] HybridClock initialized: now 1786388006768797 us; error 39 us; skew 500 ppm
I20260810 18:53:26.769126 30653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:26 dist-test-slave-6zbq krb5kdc[30639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:26.779574 30653 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:53:26.781046 30653 webserver.cc:460] Webserver started at http://127.29.174.254:44229/ using document root <none> and password file <none>
I20260810 18:53:26.781678 30653 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:26.781778 30653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:26.782027 30653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:26.785007 30653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "6eb16164f6a54925ac373c9425f19a78"
format_stamp: "Formatted at 2026-08-10 18:53:26 on dist-test-slave-6zbq"
server_key: "691e39401f0dee863b91cd62333c5fb2"
server_key_iv: "a839255f875f93681458380f35a54d23"
server_key_version: "encryptionkey@0"
I20260810 18:53:26.785708 30653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "6eb16164f6a54925ac373c9425f19a78"
format_stamp: "Formatted at 2026-08-10 18:53:26 on dist-test-slave-6zbq"
server_key: "691e39401f0dee863b91cd62333c5fb2"
server_key_iv: "a839255f875f93681458380f35a54d23"
server_key_version: "encryptionkey@0"
I20260810 18:53:26.790623 30653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:53:26.793815 30683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:26.795346 30653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:26.795486 30653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "6eb16164f6a54925ac373c9425f19a78"
format_stamp: "Formatted at 2026-08-10 18:53:26 on dist-test-slave-6zbq"
server_key: "691e39401f0dee863b91cd62333c5fb2"
server_key_iv: "a839255f875f93681458380f35a54d23"
server_key_version: "encryptionkey@0"
I20260810 18:53:26.795645 30653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:26.823693 30653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:26.827754 30653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:26.827983 30653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:26.844666 30653 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:46047
I20260810 18:53:26.844689 30766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:46047 every 8 connection(s)
I20260810 18:53:26.845891 30653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:26.849336 30770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:26.854040 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30653
I20260810 18:53:26.854542 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:26.855234 30395 external_mini_cluster.cc:1428] Setting key 4334136a3527c4ac11bbe74819167598
I20260810 18:53:26.860888 30770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: Bootstrap starting.
I20260810 18:53:26.864447 30770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:26.865621 30770 log.cc:824] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:26.868927 30770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: No bootstrap required, opened a new log
Aug 10 18:53:26 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:26.876410 30770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } }
I20260810 18:53:26.876758 30770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:26.876860 30770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb16164f6a54925ac373c9425f19a78, State: Initialized, Role: FOLLOWER
I20260810 18:53:26.877427 30770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } }
I20260810 18:53:26.877672 30770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:26.877800 30770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:26.877969 30770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:26.878161 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:26.857142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.29.174.254:46047)
0810 18:53:26.857928 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:26.857938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:26.857979 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:53:26.859809 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:26.859831 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:26.860199 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:26.860499 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:26.862265 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:26.863735 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:26.865429 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:26.866249 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:26.866468 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:26.870864 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:26.870872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:26.870889 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:26.870926 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:26.873916 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:26.874673 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:26.874679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:26.874685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:26.874862 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:26.875236 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:26.875240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:26.875242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:26.875787 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:26.876061 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:26.876509 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 18:53:26.877084 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":367,"threads_started":1}
I20260810 18:53:26.880251 30770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } }
I20260810 18:53:26.880703 30770 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb16164f6a54925ac373c9425f19a78; no voters:
I20260810 18:53:26.881244 30770 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:26.881347 30781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:26.882027 30781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb16164f6a54925ac373c9425f19a78, State: Running, Role: LEADER
I20260810 18:53:26.882568 30781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } }
I20260810 18:53:26.883251 30770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:26.885012 30782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb16164f6a54925ac373c9425f19a78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } } }
I20260810 18:53:26.885398 30782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:26.886428 30783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb16164f6a54925ac373c9425f19a78. Latest consensus state: current_term: 1 leader_uuid: "6eb16164f6a54925ac373c9425f19a78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb16164f6a54925ac373c9425f19a78" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 46047 } } }
I20260810 18:53:26.886885 30783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:26.890730 30789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:26.895645 30789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:26.905440 30789 catalog_manager.cc:1347] Generated new cluster ID: a22ffae26e8f44bf9ca11f076d6b99fc
I20260810 18:53:26.905543 30789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:26.918628 30789 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:26.919770 30789 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:26.937845 30789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: Generated new TSK 0
I20260810 18:53:26.938701 30789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.174.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
Entry for principal kudu/127.29.174.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab.
I20260810 18:53:27.014887 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:27.373443 30815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:27.373690 30815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:27.373798 30815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:27.382200 30815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:27.382309 30815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:27.382469 30815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:27.392207 30815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.193.keytab
--principal=kudu/127.29.174.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:27.393373 30815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:27.394675 30815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:27.407315 30826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:27.407358 30831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:27.407384 30827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:27.408247 30815 server_base.cc:1034] running on GCE node
I20260810 18:53:27.408834 30815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:27.409695 30815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:27.410892 30815 hybrid_clock.cc:648] HybridClock initialized: now 1786388007410848 us; error 29 us; skew 500 ppm
I20260810 18:53:27.411207 30815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:27 dist-test-slave-6zbq krb5kdc[30639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388007, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:27.414204 30815 init.cc:377] Logged in from keytab as kudu/127.29.174.193@KRBTEST.COM (short username kudu)
I20260810 18:53:27.415621 30815 webserver.cc:460] Webserver started at http://127.29.174.193:35511/ using document root <none> and password file <none>
I20260810 18:53:27.416260 30815 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:27.416370 30815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:27.416666 30815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:27.419486 30815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "5a8cba8a97514af3ad2804dbaf553b73"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "5de14791fe21cc95715b21b4563330ed"
server_key_iv: "b2c297d05f33e41719742d57c860c399"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.420218 30815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "5a8cba8a97514af3ad2804dbaf553b73"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "5de14791fe21cc95715b21b4563330ed"
server_key_iv: "b2c297d05f33e41719742d57c860c399"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.425169 30815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:53:27.428408 30842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.429677 30815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:27.429823 30815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "5a8cba8a97514af3ad2804dbaf553b73"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "5de14791fe21cc95715b21b4563330ed"
server_key_iv: "b2c297d05f33e41719742d57c860c399"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.429997 30815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:27.444350 30815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:27.448212 30815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:27.448460 30815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:27.449335 30815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:27.450623 30815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:27.450699 30815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.450814 30815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:27.450868 30815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.480288 30815 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:35729
I20260810 18:53:27.480330 31031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:35729 every 8 connection(s)
I20260810 18:53:27.481534 30815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:27.487984 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 30815
I20260810 18:53:27.488149 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:27.488550 30395 external_mini_cluster.cc:1428] Setting key 77cb6dbbd40be6bf5b710b9e7c191ac7
Aug 10 18:53:27 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388007, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.193@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:27.501467 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:27.484094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:53401 (local address 127.29.174.254:46047)
0810 18:53:27.484566 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:27.484572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:27.485499 (+ 927us) server_negotiation.cc:407] Connection header received
0810 18:53:27.487441 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:27.487445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:27.487516 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:27.487617 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:27.489129 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.489972 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.491841 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.492425 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.492554 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:27.496089 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:27.496095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:27.496099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:27.496130 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:27.498554 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:27.499346 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:27.499349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:27.499352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:27.499468 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:27.499894 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:27.499897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:27.499899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:27.500179 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:27.500308 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:27.501031 (+ 723us) server_negotiation.cc:299] Negotiation successful
0810 18:53:27.501222 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":205,"threads_started":1}
I20260810 18:53:27.502696 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:27.484667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46047 (local address 127.29.174.193:53401)
0810 18:53:27.485341 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:27.485386 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:27.487180 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:27.487824 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:27.487831 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:27.488375 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:27.488953 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:27.488970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.490168 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.490173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:27.491690 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:27.491703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.492611 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.492616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:27.492810 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:27.493413 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:27.493433 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:27.495880 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:27.498681 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:27.498690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:27.498704 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:27.499233 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:27.499560 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:27.499564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:27.499567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:27.499797 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:27.500311 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:27.500318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:27.500685 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 18:53:27.500952 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 18:53:27.501439 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":262,"threads_started":1}
I20260810 18:53:27.504106 31032 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46047
I20260810 18:53:27.504426 31032 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:27.505259 31032 heartbeater.cc:511] Master 127.29.174.254:46047 requested a full tablet report, sending...
I20260810 18:53:27.507262 30711 ts_manager.cc:194] Registered new tserver with Master: 5a8cba8a97514af3ad2804dbaf553b73 (127.29.174.193:35729)
I20260810 18:53:27.508986 30711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.193@KRBTEST.COM'} at 127.29.174.193:53401
WARNING: no policy specified for kudu/127.29.174.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
Entry for principal kudu/127.29.174.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab.
I20260810 18:53:27.554735 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:27.875766 31039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:27.876003 31039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:27.876128 31039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:27.883029 31039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:27.883119 31039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:27.883272 31039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:27.892449 31039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.194.keytab
--principal=kudu/127.29.174.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:27.893479 31039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:27.894697 31039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:27.906458 31050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:27.906428 31052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:27.906386 31054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:27.907395 31039 server_base.cc:1034] running on GCE node
I20260810 18:53:27.907927 31039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:27.908726 31039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:27.909922 31039 hybrid_clock.cc:648] HybridClock initialized: now 1786388007909881 us; error 34 us; skew 500 ppm
I20260810 18:53:27.910169 31039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:27 dist-test-slave-6zbq krb5kdc[30639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388007, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:27.913028 31039 init.cc:377] Logged in from keytab as kudu/127.29.174.194@KRBTEST.COM (short username kudu)
I20260810 18:53:27.914275 31039 webserver.cc:460] Webserver started at http://127.29.174.194:39823/ using document root <none> and password file <none>
I20260810 18:53:27.914889 31039 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:27.914988 31039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:27.915237 31039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:27.917776 31039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "7a47c342ff6942fa8cdcccd7383b2802"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "a45df6f077601b58837a95d31e48fb98"
server_key_iv: "cabe08f9d4bde690bf8d6e4e99457ca7"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.918509 31039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "7a47c342ff6942fa8cdcccd7383b2802"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "a45df6f077601b58837a95d31e48fb98"
server_key_iv: "cabe08f9d4bde690bf8d6e4e99457ca7"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.923036 31039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:27.926028 31069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.927218 31039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:27.927361 31039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "7a47c342ff6942fa8cdcccd7383b2802"
format_stamp: "Formatted at 2026-08-10 18:53:27 on dist-test-slave-6zbq"
server_key: "a45df6f077601b58837a95d31e48fb98"
server_key_iv: "cabe08f9d4bde690bf8d6e4e99457ca7"
server_key_version: "encryptionkey@0"
I20260810 18:53:27.927524 31039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:27.947053 31039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:27.950856 31039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:27.951107 31039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:27.951946 31039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:27.953132 31039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:27.953207 31039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.953325 31039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:27.953377 31039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:27.982548 31039 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:35089
I20260810 18:53:27.982591 31290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:35089 every 8 connection(s)
I20260810 18:53:27.983745 31039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:27.990154 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31039
I20260810 18:53:27.990320 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:27.990746 30395 external_mini_cluster.cc:1428] Setting key 8e77dcda5d4a3172a950bff93462d1b2
Aug 10 18:53:27 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388007, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.194@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:28.002636 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:27.986150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:53009 (local address 127.29.174.254:46047)
0810 18:53:27.986323 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:27.986328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:27.987398 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 18:53:27.989096 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:27.989102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:27.989191 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:27.989302 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:27.991033 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.991956 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.993432 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.993862 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.993974 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:27.997320 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:27.997325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:27.997328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:27.997362 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:27.999770 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.000430 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.000434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.000436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.000560 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.000980 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.000983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.000985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.001258 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.001393 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.002045 (+ 652us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.002285 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:28.003607 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:27.986581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46047 (local address 127.29.174.194:53009)
0810 18:53:27.987240 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:27.987283 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:27.988853 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:27.989533 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:27.989543 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:27.990202 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:27.990823 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:27.990844 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.992109 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.992114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:27.993285 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:27.993298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:27.994086 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:27.994095 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:27.994296 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:27.994947 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:27.994975 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:27.997098 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:27.999899 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:27.999907 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:27.999922 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.000303 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.000665 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.000670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.000673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.000865 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.001381 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:28.001389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:28.001681 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.001913 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.002409 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":255,"threads_started":1}
I20260810 18:53:28.005004 31291 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46047
I20260810 18:53:28.005271 31291 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:28.005926 31291 heartbeater.cc:511] Master 127.29.174.254:46047 requested a full tablet report, sending...
I20260810 18:53:28.007390 30711 ts_manager.cc:194] Registered new tserver with Master: 7a47c342ff6942fa8cdcccd7383b2802 (127.29.174.194:35089)
I20260810 18:53:28.008430 30711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.194@KRBTEST.COM'} at 127.29.174.194:53009
WARNING: no policy specified for kudu/127.29.174.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
Entry for principal kudu/127.29.174.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab.
I20260810 18:53:28.053494 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:28.370236 31297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:28.370487 31297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:28.370591 31297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:28.377373 31297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:28.377460 31297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:28.377576 31297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:28.386754 31297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.195.keytab
--principal=kudu/127.29.174.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:46047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:28.387831 31297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:28.388981 31297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:28.401425 31306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.401456 31308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.401485 31313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.407573 31297 server_base.cc:1034] running on GCE node
I20260810 18:53:28.408058 31297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:28.408855 31297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:28.410044 31297 hybrid_clock.cc:648] HybridClock initialized: now 1786388008409996 us; error 42 us; skew 500 ppm
I20260810 18:53:28.410322 31297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:28 dist-test-slave-6zbq krb5kdc[30639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:28.413077 31297 init.cc:377] Logged in from keytab as kudu/127.29.174.195@KRBTEST.COM (short username kudu)
I20260810 18:53:28.414376 31297 webserver.cc:460] Webserver started at http://127.29.174.195:46203/ using document root <none> and password file <none>
I20260810 18:53:28.414973 31297 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:28.415067 31297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:28.415316 31297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:28.417989 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-6zbq"
server_key: "72ca6d1906d96e442808b7b2b33fa63e"
server_key_iv: "17fc552896e347140c57f676fc74a9d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.418728 31297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-6zbq"
server_key: "72ca6d1906d96e442808b7b2b33fa63e"
server_key_iv: "17fc552896e347140c57f676fc74a9d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.423480 31297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:53:28.426627 31326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.427783 31297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:28.427901 31297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-6zbq"
server_key: "72ca6d1906d96e442808b7b2b33fa63e"
server_key_iv: "17fc552896e347140c57f676fc74a9d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.428048 31297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:28.450158 31297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:28.453691 31297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:28.453922 31297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:28.454981 31297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:28.456218 31297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:28.456298 31297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.456387 31297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:28.456429 31297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.485580 31297 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:33479
I20260810 18:53:28.485597 31514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:33479 every 8 connection(s)
I20260810 18:53:28.487054 31297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:28.488184 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31297
I20260810 18:53:28.488351 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:28.488947 30395 external_mini_cluster.cc:1428] Setting key 58e047332cf3446e02229d9899158c14
Aug 10 18:53:28 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.195@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:28.506547 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.490112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:56181 (local address 127.29.174.254:46047)
0810 18:53:28.490319 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.490326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.491612 (+ 1286us) server_negotiation.cc:407] Connection header received
0810 18:53:28.493411 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.493415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.493480 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.493557 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:28.495016 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.495787 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.497207 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.497573 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.497656 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.500885 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:28.500890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:28.500893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:28.500924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:28.503670 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.504428 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.504432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.504434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.504558 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.504957 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.504960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.504962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.505210 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.505344 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.505977 (+ 633us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.506188 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:28.507318 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.490680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:46047 (local address 127.29.174.195:56181)
0810 18:53:28.491468 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.491511 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.493175 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.493728 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.493735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.494207 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:28.494874 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.494894 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.495919 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.495922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.497097 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.497109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.497725 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.497730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.497919 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.498570 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:28.498592 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:28.500699 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:28.503791 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.503799 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.503810 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.504310 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.504659 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.504664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.504667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.504845 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.505344 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:28.505352 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:28.505622 (+ 270us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.505840 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.506282 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":304,"threads_started":1}
I20260810 18:53:28.508469 31515 heartbeater.cc:347] Connected to a master server at 127.29.174.254:46047
I20260810 18:53:28.508729 31515 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:28.509361 31515 heartbeater.cc:511] Master 127.29.174.254:46047 requested a full tablet report, sending...
I20260810 18:53:28.510741 30711 ts_manager.cc:194] Registered new tserver with Master: dd8932e58fb44bbaba3a3fc08eee4ef3 (127.29.174.195:33479)
I20260810 18:53:28.511466 30711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.174.195@KRBTEST.COM'} at 127.29.174.195:56181
I20260810 18:53:28.512181 31032 heartbeater.cc:503] Master 127.29.174.254:46047 was elected leader, sending a full tablet report...
I20260810 18:53:28.515489 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:28.530782 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.519525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.29.174.254:46047)
0810 18:53:28.519698 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.519702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.520111 (+ 409us) server_negotiation.cc:407] Connection header received
0810 18:53:28.520224 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.520228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.520284 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.520367 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:28.521689 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.522433 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.523733 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.524151 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.524243 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.526452 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:28.526457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:28.526460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:28.526486 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:28.528668 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.529210 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.529213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.529215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.529332 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.529606 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.529609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.529611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.529888 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.530067 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.530368 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.530536 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:28.534241 30711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:28.540930 30711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:28.562590 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.552006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.29.174.195:33479)
0810 18:53:28.553434 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.553442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.553478 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:53:28.553573 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.553580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.553829 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.554945 (+ 1116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:28.555224 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.556821 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.559935 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.561367 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.561565 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.561731 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.562057 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.562221 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1262,"thread_start_us":1118,"threads_started":1}
I20260810 18:53:28.563164 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.551595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:33479 (local address 127.0.0.1:52536)
0810 18:53:28.552577 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.552631 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.552822 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.554058 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.554074 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.554103 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:28.554706 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.554719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.556983 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.556988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.559480 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.559496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.561600 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.561606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.561847 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.561886 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.561997 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.562203 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":103,"spinlock_wait_cycles":5376,"thread_start_us":1414,"threads_started":1}
I20260810 18:53:28.563446 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.552346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.29.174.193:35729)
0810 18:53:28.553797 (+ 1451us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.553805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.553830 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:28.553921 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.553928 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.554148 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.555219 (+ 1071us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:28.555461 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.556937 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.559983 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.561367 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.561560 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.561731 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.562992 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.563151 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1285,"thread_start_us":1130,"threads_started":1}
I20260810 18:53:28.563760 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.551853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35729 (local address 127.0.0.1:49014)
0810 18:53:28.552577 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.552631 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.552821 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.554548 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.554553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.554581 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:28.555020 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.555033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.557071 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.557076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.559803 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.559814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.561572 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.561582 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.561828 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.561882 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.562003 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.562193 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":380,"spinlock_wait_cycles":27008,"thread_start_us":953,"threads_started":1}
I20260810 18:53:28.567116 30923 tablet_service.cc:1467] Processing CreateTablet for tablet 218a08b0ba384669ae4ade5e31f6060d (DEFAULT_TABLE table=test-table [id=977759d0e9a0469cb48373b697de0977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:28.567236 31413 tablet_service.cc:1467] Processing CreateTablet for tablet 218a08b0ba384669ae4ade5e31f6060d (DEFAULT_TABLE table=test-table [id=977759d0e9a0469cb48373b697de0977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:28.568066 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.553558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:35089 (local address 127.0.0.1:40980)
0810 18:53:28.560038 (+ 6480us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.560059 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.560189 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.561023 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.561028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.561050 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:28.561365 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.561374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.563506 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.563511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.565464 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.565479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.567373 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.567380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.567533 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.567564 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.567653 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.567763 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6332,"thread_start_us":6152,"threads_started":1}
I20260810 18:53:28.568588 30923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a08b0ba384669ae4ade5e31f6060d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:28.568636 31413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a08b0ba384669ae4ade5e31f6060d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:28.568820 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.554032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.29.174.194:35089)
0810 18:53:28.560577 (+ 6545us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.560583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.560605 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:53:28.560682 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.560687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.560848 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.561018 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:28.561825 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.563326 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.565644 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.567160 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.567328 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.567500 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.568360 (+ 860us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.568523 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6411,"thread_start_us":6275,"threads_started":1}
I20260810 18:53:28.572031 31179 tablet_service.cc:1467] Processing CreateTablet for tablet 218a08b0ba384669ae4ade5e31f6060d (DEFAULT_TABLE table=test-table [id=977759d0e9a0469cb48373b697de0977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:28.573267 31179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218a08b0ba384669ae4ade5e31f6060d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:28.579005 31544 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Bootstrap starting.
I20260810 18:53:28.582333 31548 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Bootstrap starting.
I20260810 18:53:28.582748 31544 tablet_bootstrap.cc:654] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:28.584107 31544 log.cc:824] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:28.585726 31548 tablet_bootstrap.cc:654] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:28.585901 31552 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Bootstrap starting.
I20260810 18:53:28.586918 31548 log.cc:824] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:28.588874 31552 tablet_bootstrap.cc:654] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:28.589265 31544 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: No bootstrap required, opened a new log
I20260810 18:53:28.589532 31544 ts_tablet_manager.cc:1397] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260810 18:53:28.590029 31552 log.cc:824] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:28.592059 31548 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: No bootstrap required, opened a new log
I20260810 18:53:28.592312 31548 ts_tablet_manager.cc:1397] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:53:28.592327 31552 tablet_bootstrap.cc:492] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: No bootstrap required, opened a new log
I20260810 18:53:28.592535 31552 ts_tablet_manager.cc:1397] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:53:28.599499 31552 raft_consensus.cc:348] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.599494 31544 raft_consensus.cc:348] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.599867 31552 raft_consensus.cc:374] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:28.599911 31544 raft_consensus.cc:374] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:28.599948 31552 raft_consensus.cc:729] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a47c342ff6942fa8cdcccd7383b2802, State: Initialized, Role: FOLLOWER
I20260810 18:53:28.600003 31544 raft_consensus.cc:729] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8932e58fb44bbaba3a3fc08eee4ef3, State: Initialized, Role: FOLLOWER
I20260810 18:53:28.600529 31552 consensus_queue.cc:260] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.600595 31544 consensus_queue.cc:260] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.601509 31291 heartbeater.cc:503] Master 127.29.174.254:46047 was elected leader, sending a full tablet report...
I20260810 18:53:28.602108 31552 ts_tablet_manager.cc:1428] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260810 18:53:28.603013 31515 heartbeater.cc:503] Master 127.29.174.254:46047 was elected leader, sending a full tablet report...
I20260810 18:53:28.603343 31548 raft_consensus.cc:348] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.603719 31548 raft_consensus.cc:374] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:28.603806 31548 raft_consensus.cc:729] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8cba8a97514af3ad2804dbaf553b73, State: Initialized, Role: FOLLOWER
I20260810 18:53:28.604400 31548 consensus_queue.cc:260] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:28.606839 31544 ts_tablet_manager.cc:1428] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Time spent starting tablet: real 0.017s user 0.011s sys 0.002s
I20260810 18:53:28.608631 31548 ts_tablet_manager.cc:1428] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
W20260810 18:53:28.733893 31033 tablet.cc:2369] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:28.735606 31292 tablet.cc:2369] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:28.738497 31516 tablet.cc:2369] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:29.037106 31563 raft_consensus.cc:482] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:29.037362 31563 raft_consensus.cc:504] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:29.038980 31563 leader_election.cc:288] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a8cba8a97514af3ad2804dbaf553b73 (127.29.174.193:35729), 7a47c342ff6942fa8cdcccd7383b2802 (127.29.174.194:35089)
I20260810 18:53:29.046089 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.039305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:55209 (local address 127.29.174.193:35729)
0810 18:53:29.039541 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.039547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.039803 (+ 256us) server_negotiation.cc:407] Connection header received
0810 18:53:29.040031 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.040038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.040143 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.040264 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:29.040941 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.041892 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.044235 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.045326 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.045481 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.045556 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.045766 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.045904 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:53:29.046253 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.039233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:35729 (local address 127.29.174.195:55209)
0810 18:53:29.039683 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:29.039700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:29.039854 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:29.040370 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:29.040375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:29.040405 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:29.040778 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.040791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.042022 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.042026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.044084 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.044099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.045484 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.045488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.045621 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.045646 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:53:29.045716 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:53:29.045844 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":4,"thread_start_us":225,"threads_started":1}
I20260810 18:53:29.046643 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.040067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:51749 (local address 127.29.174.194:35089)
0810 18:53:29.040253 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.040267 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.040635 (+ 368us) server_negotiation.cc:407] Connection header received
0810 18:53:29.040821 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.040825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.040890 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.040980 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:29.041443 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.042320 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.044541 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.045681 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.045830 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.045896 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.046299 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.046464 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:29.047181 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.039864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:35089 (local address 127.29.174.195:51749)
0810 18:53:29.040535 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:29.040552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:29.040690 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:29.041003 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:29.041006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:29.041026 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:29.041306 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.041313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.042479 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.042484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.044408 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.044419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.045893 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.045899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.046088 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.046124 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:29.046821 (+ 697us) client_negotiation.cc:240] Negotiation successful
0810 18:53:29.046920 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":24,"thread_start_us":386,"threads_started":1}
I20260810 18:53:29.047212 30957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a08b0ba384669ae4ade5e31f6060d" candidate_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a8cba8a97514af3ad2804dbaf553b73" is_pre_election: true
I20260810 18:53:29.048029 30957 raft_consensus.cc:2454] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8932e58fb44bbaba3a3fc08eee4ef3 in term 0.
I20260810 18:53:29.048085 31215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a08b0ba384669ae4ade5e31f6060d" candidate_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a47c342ff6942fa8cdcccd7383b2802" is_pre_election: true
I20260810 18:53:29.048516 31215 raft_consensus.cc:2454] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8932e58fb44bbaba3a3fc08eee4ef3 in term 0.
I20260810 18:53:29.048872 31328 leader_election.cc:302] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a8cba8a97514af3ad2804dbaf553b73, dd8932e58fb44bbaba3a3fc08eee4ef3; no voters:
I20260810 18:53:29.049314 31563 raft_consensus.cc:2781] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:29.049453 31563 raft_consensus.cc:482] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:29.049549 31563 raft_consensus.cc:3037] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:29.051786 31563 raft_consensus.cc:504] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:29.052364 31563 leader_election.cc:288] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [CANDIDATE]: Term 1 election: Requested vote from peers 5a8cba8a97514af3ad2804dbaf553b73 (127.29.174.193:35729), 7a47c342ff6942fa8cdcccd7383b2802 (127.29.174.194:35089)
I20260810 18:53:29.052822 30957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a08b0ba384669ae4ade5e31f6060d" candidate_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a8cba8a97514af3ad2804dbaf553b73"
I20260810 18:53:29.052858 31215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218a08b0ba384669ae4ade5e31f6060d" candidate_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a47c342ff6942fa8cdcccd7383b2802"
I20260810 18:53:29.052983 30957 raft_consensus.cc:3037] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:29.053018 31215 raft_consensus.cc:3037] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:29.055145 31215 raft_consensus.cc:2454] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8932e58fb44bbaba3a3fc08eee4ef3 in term 1.
I20260810 18:53:29.055147 30957 raft_consensus.cc:2454] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8932e58fb44bbaba3a3fc08eee4ef3 in term 1.
I20260810 18:53:29.055497 31328 leader_election.cc:302] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a8cba8a97514af3ad2804dbaf553b73, dd8932e58fb44bbaba3a3fc08eee4ef3; no voters:
I20260810 18:53:29.055761 31563 raft_consensus.cc:2781] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:29.056147 31563 raft_consensus.cc:686] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 1 LEADER]: Becoming Leader. State: Replica: dd8932e58fb44bbaba3a3fc08eee4ef3, State: Running, Role: LEADER
I20260810 18:53:29.056630 31563 consensus_queue.cc:237] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } }
I20260810 18:53:29.060886 30707 catalog_manager.cc:5495] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd8932e58fb44bbaba3a3fc08eee4ef3 (127.29.174.195). New cstate: current_term: 1 leader_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8932e58fb44bbaba3a3fc08eee4ef3" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:29.121984 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.116277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.29.174.195:33479)
0810 18:53:29.116924 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.116930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.116949 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:29.117110 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.117114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.117233 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.117369 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:29.117845 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.118546 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.119973 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.120373 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.120488 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.120924 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:29.121410 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:29.121539 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.121646 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.121728 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":418,"threads_started":1}
I20260810 18:53:29.129078 31215 raft_consensus.cc:1261] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 1 FOLLOWER]: Refusing update from remote peer dd8932e58fb44bbaba3a3fc08eee4ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:29.129312 30957 raft_consensus.cc:1261] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 1 FOLLOWER]: Refusing update from remote peer dd8932e58fb44bbaba3a3fc08eee4ef3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:29.130864 31569 consensus_queue.cc:1035] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a47c342ff6942fa8cdcccd7383b2802" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:29.131270 31563 consensus_queue.cc:1035] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a8cba8a97514af3ad2804dbaf553b73" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 35729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:29.146749 31582 mvcc.cc:205] Tried to move back new op lower bound from 7317045285384036352 to 7317045285099802624. Current Snapshot: MvccSnapshot[applied={T|T < 7317045285384036352}]
I20260810 18:53:29.156723 30711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:29.156944 30711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:29.160151 30711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6eb16164f6a54925ac373c9425f19a78: Sending DeleteTablet for 3 replicas of tablet 218a08b0ba384669ae4ade5e31f6060d
I20260810 18:53:29.161268 31413 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a08b0ba384669ae4ade5e31f6060d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:52536
I20260810 18:53:29.161742 31179 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a08b0ba384669ae4ade5e31f6060d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:40980
I20260810 18:53:29.161895 30923 tablet_service.cc:1514] Processing DeleteTablet for tablet 218a08b0ba384669ae4ade5e31f6060d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:29 UTC) from {username='kudu', principal='kudu/127.29.174.254@KRBTEST.COM'} at 127.0.0.1:49014
I20260810 18:53:29.163565 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30815
I20260810 18:53:29.174481 31609 tablet_replica.cc:330] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: stopping tablet replica
I20260810 18:53:29.175010 31609 raft_consensus.cc:2229] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:29.175572 31609 raft_consensus.cc:2258] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:29.176427 31607 tablet_replica.cc:330] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: stopping tablet replica
I20260810 18:53:29.176824 31607 raft_consensus.cc:2229] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:29.177290 31607 raft_consensus.cc:2258] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:29.178006 31609 ts_tablet_manager.cc:1905] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:29.179148 31607 ts_tablet_manager.cc:1905] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:29.183784 31606 tablet_replica.cc:330] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: stopping tablet replica
I20260810 18:53:29.185501 31609 ts_tablet_manager.cc:1918] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:29.185649 31609 log.cc:1196] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-0/wal/wals/218a08b0ba384669ae4ade5e31f6060d
I20260810 18:53:29.186192 31609 ts_tablet_manager.cc:1939] T 218a08b0ba384669ae4ade5e31f6060d P 5a8cba8a97514af3ad2804dbaf553b73: Deleting consensus metadata
I20260810 18:53:29.186167 31606 raft_consensus.cc:2229] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 1 LEADER]: Raft consensus shutting down.
Aug 10 18:53:29 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.193@KRBTEST.COM
I20260810 18:53:29.187116 31606 raft_consensus.cc:2258] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:29.188093 30687 catalog_manager.cc:4841] TS 5a8cba8a97514af3ad2804dbaf553b73 (127.29.174.193:35729): tablet 218a08b0ba384669ae4ade5e31f6060d (table test-table [id=977759d0e9a0469cb48373b697de0977]) successfully deleted
I20260810 18:53:29.188833 31606 ts_tablet_manager.cc:1905] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:29.192265 31607 ts_tablet_manager.cc:1918] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:29.192432 31607 log.cc:1196] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/218a08b0ba384669ae4ade5e31f6060d
I20260810 18:53:29.192921 31607 ts_tablet_manager.cc:1939] T 218a08b0ba384669ae4ade5e31f6060d P 7a47c342ff6942fa8cdcccd7383b2802: Deleting consensus metadata
I20260810 18:53:29.194671 30687 catalog_manager.cc:4841] TS 7a47c342ff6942fa8cdcccd7383b2802 (127.29.174.194:35089): tablet 218a08b0ba384669ae4ade5e31f6060d (table test-table [id=977759d0e9a0469cb48373b697de0977]) successfully deleted
I20260810 18:53:29.196218 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.164933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.29.174.193:35729)
0810 18:53:29.165107 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.165113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.175532 (+ 10419us) server_negotiation.cc:407] Connection header received
0810 18:53:29.175908 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.175914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.176003 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.176082 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:29.178674 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.179755 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.181635 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.182322 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.182474 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.189005 (+ 6531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:29.189019 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:29.189036 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:29.189074 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:29.192735 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.193649 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.193658 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.193661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.193827 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.194299 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.194304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.194308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.194874 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:29.195150 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.195609 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.195852 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:29.196529 31606 ts_tablet_manager.cc:1918] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:29.196673 31606 log.cc:1196] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/218a08b0ba384669ae4ade5e31f6060d
I20260810 18:53:29.197022 31025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:29.198494 31606 ts_tablet_manager.cc:1939] T 218a08b0ba384669ae4ade5e31f6060d P dd8932e58fb44bbaba3a3fc08eee4ef3: Deleting consensus metadata
I20260810 18:53:29.200388 30688 catalog_manager.cc:4841] TS dd8932e58fb44bbaba3a3fc08eee4ef3 (127.29.174.195:33479): tablet 218a08b0ba384669ae4ade5e31f6060d (table test-table [id=977759d0e9a0469cb48373b697de0977]) successfully deleted
I20260810 18:53:29.318919 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30815
I20260810 18:53:29.348402 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31039
Aug 10 18:53:29 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.194@KRBTEST.COM
I20260810 18:53:29.364235 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.349229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.29.174.194:35089)
0810 18:53:29.349455 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.349460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.349484 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:53:29.349586 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.349592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.349676 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.349763 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:29.351142 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.351852 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.353061 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.353469 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.353580 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.357691 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:29.357705 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:29.357721 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:29.357754 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:29.361112 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.362058 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.362070 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.362074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.362238 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.362652 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.362657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.362661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.363150 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:29.363467 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.363643 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.363870 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 18:53:29.364880 31282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:29.499368 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31039
I20260810 18:53:29.534122 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31297
Aug 10 18:53:29 dist-test-slave-6zbq krb5kdc[30639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.195@KRBTEST.COM
I20260810 18:53:29.557658 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.537476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52554 (local address 127.29.174.195:33479)
0810 18:53:29.537666 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.537673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.537693 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:53:29.537783 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.537789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.537871 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.537982 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:29.540058 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.541377 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.543411 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.544080 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.544218 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.549970 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:29.549987 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:29.550012 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:29.550056 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:29.554093 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.555052 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.555063 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.555067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.555251 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.555687 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.555693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.555696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.556240 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:29.556605 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.556994 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.557260 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:29.558665 31509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:29.677328 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31297
I20260810 18:53:29.703378 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 30653
I20260810 18:53:29.704404 30760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:29.811560 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 30653
2026-08-10T18:53:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4916 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T18:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:29Z Disabled control of system clock
I20260810 18:53:29.901333 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38269
--webserver_interface=127.29.174.254
--webserver_port=0
--builtin_ntp_servers=127.29.174.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:30.231475 31627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:30.231777 31627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:30.231885 31627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:30.239177 31627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:30.239283 31627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:30.239346 31627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:30.239423 31627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:30.239482 31627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:30.248646 31627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:38269
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:30.249912 31627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:30.251662 31627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:30.262756 31636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.262804 31640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.262723 31637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:30.264030 31627 server_base.cc:1034] running on GCE node
I20260810 18:53:30.264771 31627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:30.266095 31627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:30.267282 31627 hybrid_clock.cc:648] HybridClock initialized: now 1786388010267216 us; error 58 us; skew 500 ppm
I20260810 18:53:30.267611 31627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:30.269706 31627 webserver.cc:460] Webserver started at http://127.29.174.254:41833/ using document root <none> and password file <none>
I20260810 18:53:30.270332 31627 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:30.270463 31627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:30.270764 31627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:30.273505 31627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "1855d5c72941448483f605fe941093a5"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "34c9beaabf99451361bb11d287e70fa6"
server_key_iv: "bcbb2a9c55a7f79c4eb120de08a9ca61"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.274278 31627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "1855d5c72941448483f605fe941093a5"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "34c9beaabf99451361bb11d287e70fa6"
server_key_iv: "bcbb2a9c55a7f79c4eb120de08a9ca61"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.278986 31627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:53:30.282173 31652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:30.283375 31627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:30.283530 31627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "1855d5c72941448483f605fe941093a5"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "34c9beaabf99451361bb11d287e70fa6"
server_key_iv: "bcbb2a9c55a7f79c4eb120de08a9ca61"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.283717 31627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:30.294920 31627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:30.295610 31627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:30.295854 31627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:30.312700 31627 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:38269
I20260810 18:53:30.312721 31732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:38269 every 8 connection(s)
I20260810 18:53:30.313944 31627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:30.316663 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31627
I20260810 18:53:30.316844 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:30.317389 30395 external_mini_cluster.cc:1428] Setting key 1ee3948095b36f394b913bf8adcd258c
I20260810 18:53:30.318044 31734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:30.331133 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:30.319240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60104 (local address 127.29.174.254:38269)
0810 18:53:30.320168 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:30.320187 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:30.320237 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:53:30.322556 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:30.322576 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:30.323004 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:30.323316 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:30.323924 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.325269 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.327015 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.327661 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.327786 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:30.328978 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:30.328985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:30.329000 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:30.329527 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:30.329600 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:30.329621 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:30.329713 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:30.329919 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 18:53:30.330212 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":364,"threads_started":1}
I20260810 18:53:30.331168 31734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: Bootstrap starting.
I20260810 18:53:30.334097 31734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:30.335089 31734 log.cc:824] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:30.337558 31734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: No bootstrap required, opened a new log
I20260810 18:53:30.344467 31734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } }
I20260810 18:53:30.344818 31734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:30.344913 31734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1855d5c72941448483f605fe941093a5, State: Initialized, Role: FOLLOWER
I20260810 18:53:30.345434 31734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } }
I20260810 18:53:30.345657 31734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:30.345796 31734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:30.345968 31734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:30.348024 31734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } }
I20260810 18:53:30.348492 31734 leader_election.cc:302] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1855d5c72941448483f605fe941093a5; no voters:
I20260810 18:53:30.349066 31734 leader_election.cc:288] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:30.349200 31745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:30.350293 31745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [term 1 LEADER]: Becoming Leader. State: Replica: 1855d5c72941448483f605fe941093a5, State: Running, Role: LEADER
I20260810 18:53:30.350805 31745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } }
I20260810 18:53:30.351125 31734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:30.353516 31748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1855d5c72941448483f605fe941093a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } } }
I20260810 18:53:30.353916 31748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:30.357290 31751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1855d5c72941448483f605fe941093a5. Latest consensus state: current_term: 1 leader_uuid: "1855d5c72941448483f605fe941093a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1855d5c72941448483f605fe941093a5" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38269 } } }
I20260810 18:53:30.357613 31751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:30.358223 31761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:30.363678 31761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:30.372752 31761 catalog_manager.cc:1347] Generated new cluster ID: ab1b2877687d476a83b66ae2783650f2
I20260810 18:53:30.372867 31761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:30.414151 31761 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:30.415369 31761 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:30.428661 31761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: Generated new TSK 0
I20260810 18:53:30.429406 31761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:30.444240 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.193:0
--local_ip_for_outbound_sockets=127.29.174.193
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--builtin_ntp_servers=127.29.174.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:53:30.780886 31779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:30.781148 31779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:30.781248 31779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:30.788322 31779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:30.788404 31779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:30.788511 31779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.193
I20260810 18:53:30.797482 31779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.174.193
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.193
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:30.798689 31779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:30.800369 31779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:30.812486 31791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.812516 31794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.812601 31790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:30.813252 31779 server_base.cc:1034] running on GCE node
I20260810 18:53:30.813831 31779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:30.814769 31779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:30.815992 31779 hybrid_clock.cc:648] HybridClock initialized: now 1786388010815942 us; error 31 us; skew 500 ppm
I20260810 18:53:30.816308 31779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:30.818423 31779 webserver.cc:460] Webserver started at http://127.29.174.193:33137/ using document root <none> and password file <none>
I20260810 18:53:30.819049 31779 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:30.819147 31779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:30.819445 31779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:30.822192 31779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data/instance:
uuid: "7247d8f9adb94f26afdd17aa871ea67d"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "efe2ed08958dbf767aa1453335e4cd85"
server_key_iv: "7f4a7c11ce7853b6bc440ac3bfaff796"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.822993 31779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance:
uuid: "7247d8f9adb94f26afdd17aa871ea67d"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "efe2ed08958dbf767aa1453335e4cd85"
server_key_iv: "7f4a7c11ce7853b6bc440ac3bfaff796"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.827832 31779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:53:30.831190 31812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:30.832430 31779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:30.832579 31779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal
uuid: "7247d8f9adb94f26afdd17aa871ea67d"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-6zbq"
server_key: "efe2ed08958dbf767aa1453335e4cd85"
server_key_iv: "7f4a7c11ce7853b6bc440ac3bfaff796"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.832777 31779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:30.848733 31779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:30.849507 31779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:30.849787 31779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:30.850771 31779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:30.852240 31779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:30.852340 31779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:30.852450 31779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:30.852519 31779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:30.881789 31779 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.193:34775
I20260810 18:53:30.881851 31998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.193:34775 every 8 connection(s)
I20260810 18:53:30.883038 31779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:30.889348 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 31779
I20260810 18:53:30.889506 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:30.889905 30395 external_mini_cluster.cc:1428] Setting key c5c8c722bfa7955c508b6f191fcee7af
I20260810 18:53:30.892503 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.194:0
--local_ip_for_outbound_sockets=127.29.174.194
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--builtin_ntp_servers=127.29.174.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:30.895052 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:30.885334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.193:54123 (local address 127.29.174.254:38269)
0810 18:53:30.885763 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:30.885767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:30.886837 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 18:53:30.888650 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:30.888656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:30.888735 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:30.888831 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:30.889921 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.890825 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.892436 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.892870 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.892980 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:30.894230 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:30.894235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:30.894239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:30.894256 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:30.894295 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:30.894307 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:30.894424 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:30.894617 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 18:53:30.894715 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":224,"threads_started":1}
I20260810 18:53:30.895965 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:30.885755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:38269 (local address 127.29.174.193:54123)
0810 18:53:30.886664 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:30.886728 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:30.888396 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:30.889089 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:30.889096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:30.889175 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:30.889759 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:30.889777 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.890976 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.890982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:30.892296 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:30.892309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.893085 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.893094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:30.893308 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:30.893942 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:30.893944 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:30.893979 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:30.894046 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:30.894425 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:30.894437 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:30.894445 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:30.894595 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:53:30.894961 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":255,"threads_started":1}
I20260810 18:53:30.899398 32002 heartbeater.cc:347] Connected to a master server at 127.29.174.254:38269
I20260810 18:53:30.899653 32002 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:30.900331 32002 heartbeater.cc:511] Master 127.29.174.254:38269 requested a full tablet report, sending...
I20260810 18:53:30.902377 31680 ts_manager.cc:194] Registered new tserver with Master: 7247d8f9adb94f26afdd17aa871ea67d (127.29.174.193:34775)
I20260810 18:53:30.904165 31680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.193:54123
W20260810 18:53:31.215507 32010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:31.215768 32010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:31.215871 32010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:31.223078 32010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:31.223165 32010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:31.223300 32010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.194
I20260810 18:53:31.231977 32010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.174.194
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.194
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:31.233090 32010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:31.234776 32010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:31.246968 32020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:31.246935 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:31.247030 32019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:31.247865 32010 server_base.cc:1034] running on GCE node
I20260810 18:53:31.248442 32010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:31.249344 32010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:31.250556 32010 hybrid_clock.cc:648] HybridClock initialized: now 1786388011250489 us; error 48 us; skew 500 ppm
I20260810 18:53:31.250864 32010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:31.253137 32010 webserver.cc:460] Webserver started at http://127.29.174.194:35689/ using document root <none> and password file <none>
I20260810 18:53:31.253775 32010 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:31.253877 32010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:31.254158 32010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:31.256875 32010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data/instance:
uuid: "e5f2a4ead07f40b7af6e5e40067495c9"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "1fd4fba5191a43cd5122a79d576c4bb5"
server_key_iv: "b9bbf90cf18fc9ebf128835c553b8c5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.257614 32010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance:
uuid: "e5f2a4ead07f40b7af6e5e40067495c9"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "1fd4fba5191a43cd5122a79d576c4bb5"
server_key_iv: "b9bbf90cf18fc9ebf128835c553b8c5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.262557 32010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 18:53:31.265653 32037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.266863 32010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:31.267012 32010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal
uuid: "e5f2a4ead07f40b7af6e5e40067495c9"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "1fd4fba5191a43cd5122a79d576c4bb5"
server_key_iv: "b9bbf90cf18fc9ebf128835c553b8c5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.267194 32010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:31.295939 32010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:31.296694 32010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:31.296964 32010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:31.297900 32010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:31.299331 32010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:31.299425 32010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.299531 32010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:31.299603 32010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.329871 32010 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.194:35345
I20260810 18:53:31.329959 32214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.194:35345 every 8 connection(s)
I20260810 18:53:31.331064 32010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:31.331997 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32010
I20260810 18:53:31.332170 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:31.332653 30395 external_mini_cluster.cc:1428] Setting key 35fed18f333069e77b088db77d46619f
I20260810 18:53:31.335438 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.174.195:0
--local_ip_for_outbound_sockets=127.29.174.195
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--builtin_ntp_servers=127.29.174.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:53:31.343906 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.333501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:54075 (local address 127.29.174.254:38269)
0810 18:53:31.333663 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.333670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.335560 (+ 1890us) server_negotiation.cc:407] Connection header received
0810 18:53:31.337509 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.337515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.337615 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.337712 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:31.338724 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.339634 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.341209 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.341744 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.341869 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.343163 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:31.343170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:31.343174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:31.343196 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:31.343232 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:31.343248 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:31.343334 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.343503 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.343599 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:31.344727 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.334198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:38269 (local address 127.29.174.194:54075)
0810 18:53:31.335410 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.335461 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.337265 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.337888 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.337895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.337973 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:31.338560 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.338589 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.339808 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.339812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.341023 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.341036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.341954 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.341960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.342178 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.342860 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:31.342863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:31.342899 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:31.342970 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:31.343329 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:31.343335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:31.343340 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.343461 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.343728 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":468,"threads_started":1}
I20260810 18:53:31.345959 32218 heartbeater.cc:347] Connected to a master server at 127.29.174.254:38269
I20260810 18:53:31.346266 32218 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:31.347112 32218 heartbeater.cc:511] Master 127.29.174.254:38269 requested a full tablet report, sending...
I20260810 18:53:31.348503 31680 ts_manager.cc:194] Registered new tserver with Master: e5f2a4ead07f40b7af6e5e40067495c9 (127.29.174.194:35345)
I20260810 18:53:31.349535 31680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.194:54075
W20260810 18:53:31.663057 32222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:31.663302 32222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:31.663420 32222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:31.670292 32222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:31.670403 32222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:31.670555 32222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.174.195
I20260810 18:53:31.679366 32222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.174.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.174.195
--webserver_port=0
--tserver_master_addrs=127.29.174.254:38269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.174.195
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:31.680594 32222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:31.682317 32222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:31.694581 32238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:31.694644 32232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:31.694621 32233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:31.695283 32222 server_base.cc:1034] running on GCE node
I20260810 18:53:31.695822 32222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:31.696596 32222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:31.697786 32222 hybrid_clock.cc:648] HybridClock initialized: now 1786388011697744 us; error 38 us; skew 500 ppm
I20260810 18:53:31.698035 32222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:53:31.700202 32222 webserver.cc:460] Webserver started at http://127.29.174.195:46629/ using document root <none> and password file <none>
I20260810 18:53:31.700793 32222 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:31.700875 32222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:31.701138 32222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:31.703773 32222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data/instance:
uuid: "22ed73ff20fa4a20bd572dd81ec03ebf"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "5339415da9397f4ea8c12025fa0b752d"
server_key_iv: "4d72664885cd187e7d4f39974ce27fe8"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.704557 32222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance:
uuid: "22ed73ff20fa4a20bd572dd81ec03ebf"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "5339415da9397f4ea8c12025fa0b752d"
server_key_iv: "4d72664885cd187e7d4f39974ce27fe8"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.709280 32222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:31.712353 32251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.713495 32222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:53:31.713626 32222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal
uuid: "22ed73ff20fa4a20bd572dd81ec03ebf"
format_stamp: "Formatted at 2026-08-10 18:53:31 on dist-test-slave-6zbq"
server_key: "5339415da9397f4ea8c12025fa0b752d"
server_key_iv: "4d72664885cd187e7d4f39974ce27fe8"
server_key_version: "encryptionkey@0"
I20260810 18:53:31.713798 32222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:31.727376 32222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:31.728072 32222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:31.728315 32222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:31.729169 32222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:31.730508 32222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:31.730595 32222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.730674 32222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:31.730726 32222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:31.759518 32222 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.195:40815
I20260810 18:53:31.759577 32446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.195:40815 every 8 connection(s)
I20260810 18:53:31.760651 32222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:31.767416 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32222
I20260810 18:53:31.767566 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:31.767953 30395 external_mini_cluster.cc:1428] Setting key 79136b778313556482eb0a0fd0215f07
I20260810 18:53:31.772450 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.762877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.195:36993 (local address 127.29.174.254:38269)
0810 18:53:31.763034 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.763038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.764271 (+ 1233us) server_negotiation.cc:407] Connection header received
0810 18:53:31.766061 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.766067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.766150 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.766239 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:31.767297 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.768113 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.769691 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.770309 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.770452 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.771688 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:31.771695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:31.771699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:31.771718 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:31.771750 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:31.771769 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:31.771841 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.772042 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.772137 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:53:31.773293 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.763310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:38269 (local address 127.29.174.195:36993)
0810 18:53:31.764122 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.764155 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.765797 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.766479 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.766488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.766566 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:31.767092 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.767127 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.768270 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.768276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.769571 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.769583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.770548 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.770557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.770760 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.771404 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:31.771407 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:31.771442 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:31.771511 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:31.771855 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:31.771866 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:31.771873 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.772018 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.772370 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":256,"threads_started":1}
I20260810 18:53:31.774477 32447 heartbeater.cc:347] Connected to a master server at 127.29.174.254:38269
I20260810 18:53:31.774727 32447 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:31.775369 32447 heartbeater.cc:511] Master 127.29.174.254:38269 requested a full tablet report, sending...
I20260810 18:53:31.776537 31680 ts_manager.cc:194] Registered new tserver with Master: 22ed73ff20fa4a20bd572dd81ec03ebf (127.29.174.195:40815)
I20260810 18:53:31.777298 31680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.174.195:36993
I20260810 18:53:31.782333 30395 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:31.794176 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.786235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.29.174.254:38269)
0810 18:53:31.786466 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.786472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.786798 (+ 326us) server_negotiation.cc:407] Connection header received
0810 18:53:31.786942 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.786945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.787014 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.787085 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:31.787511 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.788175 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.789446 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.789826 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.789909 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.793640 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:31.793645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:31.793648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:31.793664 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:31.793694 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:31.793705 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:31.793773 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.793860 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.793928 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:31.797394 31680 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:31.804049 31680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:31.834098 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.814782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.194:35345 (local address 127.0.0.1:53088)
0810 18:53:31.821774 (+ 6992us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.821809 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.821986 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.826295 (+ 4309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.826303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.826333 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:31.826763 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.826776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.828784 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.828791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.831318 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.831337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.833222 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.833229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.833434 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.833506 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.833614 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.833788 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6740,"thread_start_us":12231,"threads_started":1}
I20260810 18:53:31.835122 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.814947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.29.174.194:35345)
0810 18:53:31.825660 (+ 10713us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.825670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.825703 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:53:31.825812 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.825820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.826064 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.826263 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:31.826980 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.828616 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.831570 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.832996 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.834407 (+ 1411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.834613 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.834681 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.834836 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10527,"thread_start_us":10398,"threads_started":1}
I20260810 18:53:31.836627 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.814909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:40815 (local address 127.0.0.1:55086)
0810 18:53:31.825898 (+ 10989us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.825917 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.826035 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.829376 (+ 3341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.829382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.829412 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:31.829881 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.829894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.832218 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.832224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.834374 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.834389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.835901 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.835907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.836076 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.836106 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.836189 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.836316 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10860,"mutex_wait_us":3,"thread_start_us":6453,"threads_started":1}
I20260810 18:53:31.837687 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.815459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.29.174.195:40815)
0810 18:53:31.828592 (+ 13133us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.828602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.828637 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:53:31.828740 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.828748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.828963 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.830105 (+ 1142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:31.830470 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.832052 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.834625 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.835714 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.835901 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.837110 (+ 1209us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.837182 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.837350 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12955,"thread_start_us":13639,"threads_started":1}
I20260810 18:53:31.839073 32118 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 (DEFAULT_TABLE table=test-table [id=85e7d58b62e34e45aa78e18c8c60e9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:31.840574 32118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9ee3b8d1644930836ef671d10faf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:31.840694 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.822265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44188)
0810 18:53:31.833656 (+ 11391us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.833671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.833812 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.834732 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.834737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.834763 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:31.835064 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.835076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.836796 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.836801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.838665 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.838683 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.839997 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.840004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.840169 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.840212 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.840298 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.840412 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11270,"thread_start_us":11133,"threads_started":1}
I20260810 18:53:31.840834 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.822376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44188 (local address 127.29.174.193:34775)
0810 18:53:31.833122 (+ 10746us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.833130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.833926 (+ 796us) server_negotiation.cc:407] Connection header received
0810 18:53:31.834026 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.834032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.834545 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.834751 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:31.835223 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.836607 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.838898 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.839811 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.839950 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.840150 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.840348 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.840495 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10590,"thread_start_us":10736,"threads_started":1}
I20260810 18:53:31.841276 32328 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 (DEFAULT_TABLE table=test-table [id=85e7d58b62e34e45aa78e18c8c60e9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:31.842715 32328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9ee3b8d1644930836ef671d10faf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:31.843747 31892 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 (DEFAULT_TABLE table=test-table [id=85e7d58b62e34e45aa78e18c8c60e9ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:31.844821 31892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9ee3b8d1644930836ef671d10faf09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:31.857714 32477 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Bootstrap starting.
I20260810 18:53:31.860767 32474 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: Bootstrap starting.
I20260810 18:53:31.861480 32475 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Bootstrap starting.
I20260810 18:53:31.862387 32477 tablet_bootstrap.cc:654] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:31.863603 32477 log.cc:824] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:31.864172 32475 tablet_bootstrap.cc:654] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:31.865010 32474 tablet_bootstrap.cc:654] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:31.865039 32475 log.cc:824] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:31.866175 32474 log.cc:824] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:31.868742 32477 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: No bootstrap required, opened a new log
I20260810 18:53:31.868976 32477 ts_tablet_manager.cc:1397] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 18:53:31.869326 32474 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: No bootstrap required, opened a new log
I20260810 18:53:31.869549 32475 tablet_bootstrap.cc:492] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: No bootstrap required, opened a new log
I20260810 18:53:31.869581 32474 ts_tablet_manager.cc:1397] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 18:53:31.869799 32475 ts_tablet_manager.cc:1397] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 18:53:31.875808 32477 raft_consensus.cc:348] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.876233 32477 raft_consensus.cc:374] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:31.876348 32477 raft_consensus.cc:729] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5f2a4ead07f40b7af6e5e40067495c9, State: Initialized, Role: FOLLOWER
I20260810 18:53:31.876929 32477 consensus_queue.cc:260] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.878947 32218 heartbeater.cc:503] Master 127.29.174.254:38269 was elected leader, sending a full tablet report...
I20260810 18:53:31.879674 32474 raft_consensus.cc:348] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.880146 32474 raft_consensus.cc:374] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:31.880266 32474 raft_consensus.cc:729] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7247d8f9adb94f26afdd17aa871ea67d, State: Initialized, Role: FOLLOWER
I20260810 18:53:31.880579 32477 ts_tablet_manager.cc:1428] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 18:53:31.879907 32475 raft_consensus.cc:348] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.880950 32475 raft_consensus.cc:374] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:31.880815 32474 consensus_queue.cc:260] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.881099 32475 raft_consensus.cc:729] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ed73ff20fa4a20bd572dd81ec03ebf, State: Initialized, Role: FOLLOWER
I20260810 18:53:31.881749 32475 consensus_queue.cc:260] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.885916 32002 heartbeater.cc:503] Master 127.29.174.254:38269 was elected leader, sending a full tablet report...
I20260810 18:53:31.886425 32447 heartbeater.cc:503] Master 127.29.174.254:38269 was elected leader, sending a full tablet report...
I20260810 18:53:31.887121 32475 ts_tablet_manager.cc:1428] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260810 18:53:31.888104 32474 ts_tablet_manager.cc:1428] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
I20260810 18:53:31.889653 32485 raft_consensus.cc:482] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:31.890913 32485 raft_consensus.cc:504] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.892499 32485 leader_election.cc:288] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ed73ff20fa4a20bd572dd81ec03ebf (127.29.174.195:40815), 7247d8f9adb94f26afdd17aa871ea67d (127.29.174.193:34775)
I20260810 18:53:31.900218 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.892766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:34057 (local address 127.29.174.195:40815)
0810 18:53:31.892990 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.892997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.893532 (+ 535us) server_negotiation.cc:407] Connection header received
0810 18:53:31.893901 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.893908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.893985 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.894047 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:31.894817 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.895727 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.898177 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.899338 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.899493 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.899563 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.899880 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.900004 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:53:31.900331 32488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.892682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.195:40815 (local address 127.29.174.194:34057)
0810 18:53:31.893446 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.893465 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.893645 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.894231 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.894235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.894261 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:31.894641 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.894654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.895863 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.895866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.897992 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.898007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.899503 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.899509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.899721 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.899757 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.899853 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.899977 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":105,"thread_start_us":473,"threads_started":1}
I20260810 18:53:31.900410 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.893852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.174.194:45539 (local address 127.29.174.193:34775)
0810 18:53:31.894058 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.894063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.894190 (+ 127us) server_negotiation.cc:407] Connection header received
0810 18:53:31.894446 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.894452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.894537 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.894626 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:31.895255 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.896134 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.898367 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.899492 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.899621 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.899678 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.900020 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.900146 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 18:53:31.900954 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.893542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.29.174.194:45539)
0810 18:53:31.894104 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:31.894120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:31.894296 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:31.894667 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:31.894672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:31.894697 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:31.895076 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.895089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.896266 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.896272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.898193 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:31.898206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.899649 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.899655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:31.899860 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.899888 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:53:31.900592 (+ 704us) client_negotiation.cc:240] Negotiation successful
0810 18:53:31.900727 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":20,"thread_start_us":523,"threads_started":1}
I20260810 18:53:31.901326 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9ee3b8d1644930836ef671d10faf09" candidate_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" is_pre_election: true
I20260810 18:53:31.901819 32359 raft_consensus.cc:2454] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5f2a4ead07f40b7af6e5e40067495c9 in term 0.
I20260810 18:53:31.901918 31928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9ee3b8d1644930836ef671d10faf09" candidate_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7247d8f9adb94f26afdd17aa871ea67d" is_pre_election: true
I20260810 18:53:31.902359 31928 raft_consensus.cc:2454] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5f2a4ead07f40b7af6e5e40067495c9 in term 0.
I20260810 18:53:31.902453 32044 leader_election.cc:302] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ed73ff20fa4a20bd572dd81ec03ebf, e5f2a4ead07f40b7af6e5e40067495c9; no voters:
I20260810 18:53:31.902799 32485 raft_consensus.cc:2781] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:31.902943 32485 raft_consensus.cc:482] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:31.903051 32485 raft_consensus.cc:3037] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:31.905069 32485 raft_consensus.cc:504] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.905591 32485 leader_election.cc:288] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [CANDIDATE]: Term 1 election: Requested vote from peers 22ed73ff20fa4a20bd572dd81ec03ebf (127.29.174.195:40815), 7247d8f9adb94f26afdd17aa871ea67d (127.29.174.193:34775)
I20260810 18:53:31.905951 32359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9ee3b8d1644930836ef671d10faf09" candidate_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf"
I20260810 18:53:31.906060 31928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9ee3b8d1644930836ef671d10faf09" candidate_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7247d8f9adb94f26afdd17aa871ea67d"
I20260810 18:53:31.906118 32359 raft_consensus.cc:3037] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:31.906241 31928 raft_consensus.cc:3037] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:31.908215 32359 raft_consensus.cc:2454] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5f2a4ead07f40b7af6e5e40067495c9 in term 1.
I20260810 18:53:31.908552 31928 raft_consensus.cc:2454] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5f2a4ead07f40b7af6e5e40067495c9 in term 1.
I20260810 18:53:31.908583 32044 leader_election.cc:302] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ed73ff20fa4a20bd572dd81ec03ebf, e5f2a4ead07f40b7af6e5e40067495c9; no voters:
I20260810 18:53:31.908869 32485 raft_consensus.cc:2781] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:31.909286 32485 raft_consensus.cc:686] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 1 LEADER]: Becoming Leader. State: Replica: e5f2a4ead07f40b7af6e5e40067495c9, State: Running, Role: LEADER
I20260810 18:53:31.909673 32485 consensus_queue.cc:237] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } }
I20260810 18:53:31.913900 31677 catalog_manager.cc:5495] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5f2a4ead07f40b7af6e5e40067495c9 (127.29.174.194). New cstate: current_term: 1 leader_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5f2a4ead07f40b7af6e5e40067495c9" member_type: VOTER last_known_addr { host: "127.29.174.194" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:31.946722 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.941542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.29.174.194:35345)
0810 18:53:31.941749 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.941755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.941819 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:53:31.942051 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.942055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:31.942130 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:31.942261 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:31.942639 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.943326 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.944687 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:31.945094 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:31.945202 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:31.945682 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:31.946141 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:31.946266 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:31.946412 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:53:31.946486 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:31.954048 31928 raft_consensus.cc:1261] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d [term 1 FOLLOWER]: Refusing update from remote peer e5f2a4ead07f40b7af6e5e40067495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:31.954066 32359 raft_consensus.cc:1261] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 1 FOLLOWER]: Refusing update from remote peer e5f2a4ead07f40b7af6e5e40067495c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:31.954923 32491 consensus_queue.cc:1035] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ed73ff20fa4a20bd572dd81ec03ebf" member_type: VOTER last_known_addr { host: "127.29.174.195" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:31.955245 32485 consensus_queue.cc:1035] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7247d8f9adb94f26afdd17aa871ea67d" member_type: VOTER last_known_addr { host: "127.29.174.193" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:31.981799 31677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:31.982028 31677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:31.985409 31677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1855d5c72941448483f605fe941093a5: Sending DeleteTablet for 3 replicas of tablet 0b9ee3b8d1644930836ef671d10faf09
I20260810 18:53:31.986394 32118 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:31 UTC) from {username='slave'} at 127.0.0.1:53088
I20260810 18:53:31.986685 32328 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:31 UTC) from {username='slave'} at 127.0.0.1:55086
I20260810 18:53:31.987067 31892 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9ee3b8d1644930836ef671d10faf09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:31 UTC) from {username='slave'} at 127.0.0.1:44188
I20260810 18:53:31.987699 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31779
I20260810 18:53:32.008173 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:31.988818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.29.174.193:34775)
0810 18:53:31.988995 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:31.989002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:31.999457 (+ 10455us) server_negotiation.cc:407] Connection header received
0810 18:53:31.999876 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:31.999883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:32.000011 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:32.000084 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:32.000728 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.002006 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.004007 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.004699 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.004805 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.006386 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:32.006400 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:32.006419 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:32.007201 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:32.007263 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:32.007289 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:32.007395 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:32.007649 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 18:53:32.007785 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:32.009419 31991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:32.015091 32514 tablet_replica.cc:330] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: stopping tablet replica
I20260810 18:53:32.015619 32514 raft_consensus.cc:2229] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:32.016916 32514 raft_consensus.cc:2258] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:32.017094 32516 tablet_replica.cc:330] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: stopping tablet replica
I20260810 18:53:32.017867 32516 raft_consensus.cc:2229] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:32.018376 32516 raft_consensus.cc:2258] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:32.018862 32514 ts_tablet_manager.cc:1905] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:32.020208 32516 ts_tablet_manager.cc:1905] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:32.026185 32514 ts_tablet_manager.cc:1918] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:32.026374 32514 log.cc:1196] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-1/wal/wals/0b9ee3b8d1644930836ef671d10faf09
I20260810 18:53:32.026942 32514 ts_tablet_manager.cc:1939] T 0b9ee3b8d1644930836ef671d10faf09 P e5f2a4ead07f40b7af6e5e40067495c9: Deleting consensus metadata
I20260810 18:53:32.026944 32516 ts_tablet_manager.cc:1918] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:32.027086 32516 log.cc:1196] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Deleting WAL directory at /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387695213813-30395-0/minicluster-data/ts-2/wal/wals/0b9ee3b8d1644930836ef671d10faf09
I20260810 18:53:32.027612 32516 ts_tablet_manager.cc:1939] T 0b9ee3b8d1644930836ef671d10faf09 P 22ed73ff20fa4a20bd572dd81ec03ebf: Deleting consensus metadata
I20260810 18:53:32.028632 31657 catalog_manager.cc:4841] TS e5f2a4ead07f40b7af6e5e40067495c9 (127.29.174.194:35345): tablet 0b9ee3b8d1644930836ef671d10faf09 (table test-table [id=85e7d58b62e34e45aa78e18c8c60e9ea]) successfully deleted
I20260810 18:53:32.029816 31658 catalog_manager.cc:4841] TS 22ed73ff20fa4a20bd572dd81ec03ebf (127.29.174.195:40815): tablet 0b9ee3b8d1644930836ef671d10faf09 (table test-table [id=85e7d58b62e34e45aa78e18c8c60e9ea]) successfully deleted
I20260810 18:53:32.103972 32517 tablet_replica.cc:330] T 0b9ee3b8d1644930836ef671d10faf09 P 7247d8f9adb94f26afdd17aa871ea67d: stopping tablet replica
I20260810 18:53:32.104211 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31779
W20260810 18:53:32.127461 31658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.174.193:34775 (error 108)
I20260810 18:53:32.128775 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32010
I20260810 18:53:32.129383 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.128842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44216)
0810 18:53:32.129090 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.129213 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260810 18:53:32.129801 31658 catalog_manager.cc:4584] TS 7247d8f9adb94f26afdd17aa871ea67d (127.29.174.193:34775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0b9ee3b8d1644930836ef671d10faf09: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
I20260810 18:53:32.136230 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.129650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.29.174.194:35345)
0810 18:53:32.129847 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:32.129853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:32.129874 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:32.129940 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:32.129946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:32.130050 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:32.130163 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:32.130583 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.131655 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.132903 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.133538 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.133663 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.134688 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:32.134700 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:32.134718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:32.135487 (+ 769us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:32.135550 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:32.135575 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:32.135686 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:32.135757 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 18:53:32.135859 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:32.136961 32207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:32.183365 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.182905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44226)
0810 18:53:32.183111 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.183201 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:53:32.184275 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.183965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44234)
0810 18:53:32.184090 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.184160 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 18:53:32.243762 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32010
I20260810 18:53:32.245517 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.245075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44246)
0810 18:53:32.245280 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.245365 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:53:32.246500 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.246120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44250)
0810 18:53:32.246269 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.246373 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 18:53:32.264497 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32222
I20260810 18:53:32.271054 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.265195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.29.174.195:40815)
0810 18:53:32.265380 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:32.265387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:32.265406 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:32.265573 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:32.265577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:32.265669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:32.265753 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:32.266135 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.266923 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.268177 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.268574 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.268688 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.269675 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:32.269684 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:32.269698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:32.270246 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:32.270292 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:32.270312 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:32.270532 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:32.270664 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:53:32.270767 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:32.271670 32438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:32.327031 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.326592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44266)
0810 18:53:32.326791 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.326882 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:53:32.327998 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.327636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.193:34775 (local address 127.0.0.1:44278)
0810 18:53:32.327832 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.327899 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.193:34775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 18:53:32.363279 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32222
I20260810 18:53:32.381564 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 31627
I20260810 18:53:32.382560 31727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:32.490113 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 31627
2026-08-10T18:53:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2685 ms)
[----------] 6 tests from EncryptionPolicyTest (21830 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:32 dist-test-slave-6zbq krb5kdc[32533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:32 dist-test-slave-6zbq krb5kdc[32533](info): set up 2 sockets
Aug 10 18:53:32 dist-test-slave-6zbq krb5kdc[32533](info): commencing operation
krb5kdc: starting...
W20260810 18:53:33.747457 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:33 dist-test-slave-6zbq krb5kdc[32533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:34.029641 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:34.379074 32551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:34.379364 32551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:34.379519 32551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:34.386617 32551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:34.386731 32551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:34.386804 32551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:34.386885 32551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:34.386945 32551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:34.396983 32551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:34.398221 32551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:34.399515 32551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:34.410591 32568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:34.410666 32565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:34.410691 32564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:34.411568 32551 server_base.cc:1034] running on GCE node
I20260810 18:53:34.412252 32551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:34.413502 32551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:34.414700 32551 hybrid_clock.cc:648] HybridClock initialized: now 1786388014414657 us; error 31 us; skew 500 ppm
I20260810 18:53:34.415012 32551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:34 dist-test-slave-6zbq krb5kdc[32533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388014, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:34.417850 32551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:34.419768 32551 webserver.cc:460] Webserver started at http://127.0.0.1:35295/ using document root <none> and password file <none>
I20260810 18:53:34.420378 32551 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:34.420495 32551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:34.420781 32551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:34.423614 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "89f915f69bfd4467aa2a36a757233fa3"
format_stamp: "Formatted at 2026-08-10 18:53:34 on dist-test-slave-6zbq"
server_key: "89f71624d5441f03c5bc7c04e0b04ade"
server_key_iv: "677ad3301b67629998318ab6c7260d6c"
server_key_version: "encryptionkey@0"
I20260810 18:53:34.424368 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "89f915f69bfd4467aa2a36a757233fa3"
format_stamp: "Formatted at 2026-08-10 18:53:34 on dist-test-slave-6zbq"
server_key: "89f71624d5441f03c5bc7c04e0b04ade"
server_key_iv: "677ad3301b67629998318ab6c7260d6c"
server_key_version: "encryptionkey@0"
I20260810 18:53:34.429032 32551 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:34.432410 32579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:34.433590 32551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:34.433751 32551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "89f915f69bfd4467aa2a36a757233fa3"
format_stamp: "Formatted at 2026-08-10 18:53:34 on dist-test-slave-6zbq"
server_key: "89f71624d5441f03c5bc7c04e0b04ade"
server_key_iv: "677ad3301b67629998318ab6c7260d6c"
server_key_version: "encryptionkey@0"
I20260810 18:53:34.433918 32551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:34.447562 32551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:34.451196 32551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:34.451453 32551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:34.468305 32551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43013
I20260810 18:53:34.468348 32664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43013 every 8 connection(s)
I20260810 18:53:34.469595 32551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:34.473085 32667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:34.477900 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32551
I20260810 18:53:34.478094 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:34.478796 30395 external_mini_cluster.cc:1428] Setting key a3dd3c0eff6e3529ef96562eca9a60f4
I20260810 18:53:34.483764 32667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3: Bootstrap starting.
I20260810 18:53:34.487349 32667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:34.488570 32667 log.cc:824] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:34.491547 32667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3: No bootstrap required, opened a new log
Aug 10 18:53:34 dist-test-slave-6zbq krb5kdc[32533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:53:34.500115 32667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260810 18:53:34.500473 32667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:34.500567 32667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f915f69bfd4467aa2a36a757233fa3, State: Initialized, Role: FOLLOWER
I20260810 18:53:34.501145 32667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260810 18:53:34.501379 32667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:34.501523 32667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:34.501686 32667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:34.502736 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:34.480684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38371 (local address 127.0.0.1:43013)
0810 18:53:34.481567 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:34.481582 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:34.481926 (+ 344us) server_negotiation.cc:407] Connection header received
0810 18:53:34.484268 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:34.484299 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:34.484744 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:34.485141 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:34.486395 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:34.488350 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:34.489877 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:34.490889 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:34.491075 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:34.495260 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:34.495268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:34.495281 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:34.495319 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:34.498558 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:34.499484 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:34.499489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:34.499495 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:34.499639 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:34.500107 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:34.500110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:34.500112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:34.500632 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:34.500931 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:34.501433 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 18:53:34.501746 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":387,"threads_started":1}
I20260810 18:53:34.504030 32667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260810 18:53:34.504552 32667 leader_election.cc:302] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f915f69bfd4467aa2a36a757233fa3; no voters:
I20260810 18:53:34.505156 32667 leader_election.cc:288] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:34.505264 32681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:34.506515 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:34.506547 32681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 89f915f69bfd4467aa2a36a757233fa3, State: Running, Role: LEADER
I20260810 18:53:34.507073 32681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260810 18:53:34.507268 32667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:34.509932 32684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f915f69bfd4467aa2a36a757233fa3. Latest consensus state: current_term: 1 leader_uuid: "89f915f69bfd4467aa2a36a757233fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260810 18:53:34.510322 32684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:34.511436 32682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f915f69bfd4467aa2a36a757233fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f915f69bfd4467aa2a36a757233fa3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260810 18:53:34.513223 32682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:34.521552 32693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:34.528205 32693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:34.533306 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:34.515265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55661 (local address 127.0.0.1:43013)
0810 18:53:34.515433 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:34.515439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:34.515458 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:34.515511 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:34.515516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:34.515587 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:34.515678 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:34.517763 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:34.519144 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:34.521055 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:34.521781 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:34.521910 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:34.524824 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:34.524832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:34.524841 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:34.524880 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:34.528881 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:34.529938 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:34.529944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:34.529949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:34.530101 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:34.530707 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:34.530714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:34.530718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:34.531148 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:34.531460 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:34.532704 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0810 18:53:34.532904 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:34.537875 32693 catalog_manager.cc:1347] Generated new cluster ID: 8b8aae1ca90040c69e8d25ab1bf32f7c
I20260810 18:53:34.537979 32693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:34.569373 32693 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:34.570561 32693 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:34.577561 32693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f915f69bfd4467aa2a36a757233fa3: Generated new TSK 0
I20260810 18:53:34.578233 32693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:34.590181 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32551
I20260810 18:53:34.591321 32658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:34.683187 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32551
W20260810 18:53:34.706171 32529 connection.cc:535] client connection to 127.0.0.1:43013 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43013): Connection reset by peer (error 104)
2026-08-10T18:53:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:34 dist-test-slave-6zbq krb5kdc[32726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:34 dist-test-slave-6zbq krb5kdc[32726](info): set up 2 sockets
Aug 10 18:53:34 dist-test-slave-6zbq krb5kdc[32726](info): commencing operation
krb5kdc: starting...
W20260810 18:53:35.913805 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:36 dist-test-slave-6zbq krb5kdc[32726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:36.166415 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:36.484413 32746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:36.484673 32746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:36.484820 32746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:36.491665 32746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:36.491772 32746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:36.491855 32746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:36.491931 32746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:36.491987 32746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:36.501113 32746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:36.502188 32746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:36.503391 32746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:36.514536 32753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:36.514518 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:36.514575 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:36.515283 32746 server_base.cc:1034] running on GCE node
I20260810 18:53:36.516628 32746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:36.517830 32746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:36.519025 32746 hybrid_clock.cc:648] HybridClock initialized: now 1786388016518981 us; error 30 us; skew 500 ppm
I20260810 18:53:36.519330 32746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:36 dist-test-slave-6zbq krb5kdc[32726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388016, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:36.522245 32746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:36.523586 32746 webserver.cc:460] Webserver started at http://127.0.0.1:41135/ using document root <none> and password file <none>
I20260810 18:53:36.524178 32746 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:36.524296 32746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:36.524602 32746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:36.527442 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "caebcfde71b14c29a58c356c01c434f7"
format_stamp: "Formatted at 2026-08-10 18:53:36 on dist-test-slave-6zbq"
server_key: "531331fb2683a73f6f055969e6ad8b42"
server_key_iv: "b38c503d21ae92f8a9c1a4581a937381"
server_key_version: "encryptionkey@0"
I20260810 18:53:36.528250 32746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "caebcfde71b14c29a58c356c01c434f7"
format_stamp: "Formatted at 2026-08-10 18:53:36 on dist-test-slave-6zbq"
server_key: "531331fb2683a73f6f055969e6ad8b42"
server_key_iv: "b38c503d21ae92f8a9c1a4581a937381"
server_key_version: "encryptionkey@0"
I20260810 18:53:36.532966 32746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 18:53:36.536288 32764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:36.537480 32746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:36.537626 32746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "caebcfde71b14c29a58c356c01c434f7"
format_stamp: "Formatted at 2026-08-10 18:53:36 on dist-test-slave-6zbq"
server_key: "531331fb2683a73f6f055969e6ad8b42"
server_key_iv: "b38c503d21ae92f8a9c1a4581a937381"
server_key_version: "encryptionkey@0"
I20260810 18:53:36.537803 32746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:36.560927 32746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:36.564980 32746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:36.565236 32746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:36.582607 32746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37135
I20260810 18:53:36.582643 396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37135 every 8 connection(s)
I20260810 18:53:36.583896 32746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:36.587443 397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:36.587469 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 32746
I20260810 18:53:36.587636 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:36.588059 30395 external_mini_cluster.cc:1428] Setting key 79391bd10ca98d15452f7343cc87a168
I20260810 18:53:36.602006 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: Bootstrap starting.
I20260810 18:53:36.602044 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:36.589717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57009 (local address 127.0.0.1:37135)
0810 18:53:36.590612 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:36.590629 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:36.590682 (+ 53us) server_negotiation.cc:407] Connection header received
0810 18:53:36.592386 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:36.592412 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:36.592888 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:36.593304 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:36.593811 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:36.595800 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:36.597091 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:36.598097 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:36.598309 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:36.599494 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:36.599505 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:36.599526 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:36.600205 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:36.600294 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:36.600324 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:36.600466 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:36.600655 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:53:36.600831 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":358,"threads_started":1}
I20260810 18:53:36.605598 397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:36.606947 397 log.cc:824] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:36.606948 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:36.615119 397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: No bootstrap required, opened a new log
I20260810 18:53:36.623381 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:36.613113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41753 (local address 127.0.0.1:37135)
0810 18:53:36.615618 (+ 2505us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:36.615623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:36.615647 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:53:36.615873 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:36.615879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:36.615958 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:36.616065 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:36.616987 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:36.618139 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:36.620047 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:36.620754 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:36.620892 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:36.622386 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:36.622394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:36.622398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:36.622421 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:36.622493 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:36.622516 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:36.622636 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:36.622998 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:53:36.623053 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2385}
I20260810 18:53:36.625797 397 raft_consensus.cc:348] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260810 18:53:36.626168 397 raft_consensus.cc:374] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:36.626233 397 raft_consensus.cc:729] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caebcfde71b14c29a58c356c01c434f7, State: Initialized, Role: FOLLOWER
I20260810 18:53:36.626890 397 consensus_queue.cc:260] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260810 18:53:36.627177 397 raft_consensus.cc:388] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:36.627332 397 raft_consensus.cc:482] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:36.627509 397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:36.630674 397 raft_consensus.cc:504] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260810 18:53:36.631266 397 leader_election.cc:302] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caebcfde71b14c29a58c356c01c434f7; no voters:
I20260810 18:53:36.631901 397 leader_election.cc:288] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:36.632038 414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:36.633378 414 raft_consensus.cc:686] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [term 1 LEADER]: Becoming Leader. State: Replica: caebcfde71b14c29a58c356c01c434f7, State: Running, Role: LEADER
I20260810 18:53:36.633986 414 consensus_queue.cc:237] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260810 18:53:36.634147 397 sys_catalog.cc:564] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:36.643865 416 sys_catalog.cc:455] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caebcfde71b14c29a58c356c01c434f7. Latest consensus state: current_term: 1 leader_uuid: "caebcfde71b14c29a58c356c01c434f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } } }
I20260810 18:53:36.644212 416 sys_catalog.cc:458] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:36.644448 415 sys_catalog.cc:455] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caebcfde71b14c29a58c356c01c434f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caebcfde71b14c29a58c356c01c434f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } } }
I20260810 18:53:36.644752 415 sys_catalog.cc:458] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7 [sys.catalog]: This master's current role is: LEADER
W20260810 18:53:36.648940 438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:36.649091 438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:36.651757 439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:36.653887 439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:36.662096 439 catalog_manager.cc:1347] Generated new cluster ID: 1b1e1e1b7e0840a38e8938707e48d8b5
I20260810 18:53:36.662190 439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:36.699261 439 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:36.700690 439 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:36.711946 439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caebcfde71b14c29a58c356c01c434f7: Generated new TSK 0
I20260810 18:53:36.712728 439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:36.731107 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32746
W20260810 18:53:36.732597 390 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57009
W20260810 18:53:36.733240 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 32746: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:36.733392 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 32746
W20260810 18:53:36.755322 32723 connection.cc:535] client connection to 127.0.0.1:37135 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37135): Connection reset by peer (error 104)
2026-08-10T18:53:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2048 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:36 dist-test-slave-6zbq krb5kdc[448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:36 dist-test-slave-6zbq krb5kdc[448](info): set up 2 sockets
Aug 10 18:53:36 dist-test-slave-6zbq krb5kdc[448](info): commencing operation
krb5kdc: starting...
W20260810 18:53:37.968847 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:38 dist-test-slave-6zbq krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:38.208941 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:38.529052 469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:38.529337 469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:38.529477 469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:38.536329 469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:38.536444 469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:38.536522 469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:38.536607 469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:38.536669 469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:38.546080 469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:38.547273 469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:38.548462 469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:38.559212 476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.559320 478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.559592 475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:38.560251 469 server_base.cc:1034] running on GCE node
I20260810 18:53:38.560950 469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:38.562184 469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:38.563401 469 hybrid_clock.cc:648] HybridClock initialized: now 1786388018563344 us; error 47 us; skew 500 ppm
I20260810 18:53:38.563726 469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:38 dist-test-slave-6zbq krb5kdc[448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:38.566684 469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:38.568207 469 webserver.cc:460] Webserver started at http://127.0.0.1:32913/ using document root <none> and password file <none>
I20260810 18:53:38.568817 469 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:38.568936 469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:38.569202 469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:38.572117 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-6zbq"
server_key: "8082bf648a0a275642c7fa0ca5d093bd"
server_key_iv: "587f409001e9ab4d49029964115de1d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.572851 469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-6zbq"
server_key: "8082bf648a0a275642c7fa0ca5d093bd"
server_key_iv: "587f409001e9ab4d49029964115de1d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.577620 469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:38.580817 487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.581982 469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:38.582161 469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-6zbq"
server_key: "8082bf648a0a275642c7fa0ca5d093bd"
server_key_iv: "587f409001e9ab4d49029964115de1d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.582331 469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:38.587682 469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:38.591216 469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:38.591467 469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:38.611697 469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33193
I20260810 18:53:38.611744 580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33193 every 8 connection(s)
I20260810 18:53:38.612950 469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:38.616394 584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:38.620476 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 469
I20260810 18:53:38.620633 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:38.621059 30395 external_mini_cluster.cc:1428] Setting key aaa8954ea0200d7c68edd0268ffab997
I20260810 18:53:38.628067 584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b: Bootstrap starting.
I20260810 18:53:38.630098 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.622686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39283 (local address 127.0.0.1:33193)
0810 18:53:38.623486 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.623500 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.623547 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:53:38.626207 (+ 2660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.626235 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.626685 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.627109 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.628081 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.628089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.628105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:38.628124 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.628311 (+ 187us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:38.628352 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.628584 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.628734 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.628911 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":296,"threads_started":1}
I20260810 18:53:38.631748 584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:38.632862 584 log.cc:824] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:38.633567 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:38.635798 584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b: No bootstrap required, opened a new log
I20260810 18:53:38.640652 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.638563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58283 (local address 127.0.0.1:33193)
0810 18:53:38.638735 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.638740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.639005 (+ 265us) server_negotiation.cc:407] Connection header received
0810 18:53:38.639133 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.639138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.639192 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.639308 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.640053 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.640058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.640063 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:38.640080 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.640123 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:38.640141 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.640271 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.640313 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.640372 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:38.644070 584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:53:38.644371 584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:38.644446 584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5d9cc9cebc4dc582d345dd2e3bdf5b, State: Initialized, Role: FOLLOWER
I20260810 18:53:38.644930 584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:53:38.645149 584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:38.645280 584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:38.645427 584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:38.647661 584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:53:38.648113 584 leader_election.cc:302] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c5d9cc9cebc4dc582d345dd2e3bdf5b; no voters:
I20260810 18:53:38.648626 584 leader_election.cc:288] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:38.648753 604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:38.649405 604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [term 1 LEADER]: Becoming Leader. State: Replica: 0c5d9cc9cebc4dc582d345dd2e3bdf5b, State: Running, Role: LEADER
I20260810 18:53:38.649900 604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:53:38.650449 584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:38.652627 605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } } }
I20260810 18:53:38.653041 605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:38.652858 606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c5d9cc9cebc4dc582d345dd2e3bdf5b. Latest consensus state: current_term: 1 leader_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c5d9cc9cebc4dc582d345dd2e3bdf5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } } }
I20260810 18:53:38.653177 606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:38.655613 619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:38.659641 619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:38.667433 619 catalog_manager.cc:1347] Generated new cluster ID: c45a0504a7754cb0be0758b9a1876a42
I20260810 18:53:38.667522 619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:38.695571 619 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:38.695964 619 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:38.704159 619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c5d9cc9cebc4dc582d345dd2e3bdf5b: Generated new TSK 0
I20260810 18:53:38.704821 619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:38.710464 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 469
W20260810 18:53:38.711453 572 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39283
W20260810 18:53:38.712196 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 469: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:38.712345 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 469
2026-08-10T18:53:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:38 dist-test-slave-6zbq krb5kdc[639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:38 dist-test-slave-6zbq krb5kdc[639](info): set up 2 sockets
Aug 10 18:53:38 dist-test-slave-6zbq krb5kdc[639](info): commencing operation
krb5kdc: starting...
W20260810 18:53:39.964416 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:40.231689 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:40.560839 664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:40.561110 664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:40.561209 664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:40.568207 664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:40.568303 664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:40.568361 664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:40.568420 664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:40.568463 664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:40.577806 664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:40.578980 664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:40.580178 664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:40.591163 680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.591140 677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.591745 672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:40.591905 664 server_base.cc:1034] running on GCE node
I20260810 18:53:40.592660 664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:40.593899 664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:40.595129 664 hybrid_clock.cc:648] HybridClock initialized: now 1786388020595072 us; error 35 us; skew 500 ppm
I20260810 18:53:40.595471 664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388020, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:40.598444 664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:40.599804 664 webserver.cc:460] Webserver started at http://127.0.0.1:45691/ using document root <none> and password file <none>
I20260810 18:53:40.600409 664 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:40.600533 664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:40.600817 664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:40.603842 664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "089363ee41db44c5a710209ef1f72b35"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-6zbq"
server_key: "dc035713e058ec637cda1c83804ac509"
server_key_iv: "ba31cb11e41719a83e4008238d08ea0d"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.604864 664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "089363ee41db44c5a710209ef1f72b35"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-6zbq"
server_key: "dc035713e058ec637cda1c83804ac509"
server_key_iv: "ba31cb11e41719a83e4008238d08ea0d"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.612250 664 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260810 18:53:40.615722 689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:40.616883 664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:40.617043 664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "089363ee41db44c5a710209ef1f72b35"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-6zbq"
server_key: "dc035713e058ec637cda1c83804ac509"
server_key_iv: "ba31cb11e41719a83e4008238d08ea0d"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.617215 664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:40.633056 664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:40.636554 664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:40.636795 664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:40.653681 664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33989
I20260810 18:53:40.653723 785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33989 every 8 connection(s)
I20260810 18:53:40.655182 664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:40.658824 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:40.659049 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 664
I20260810 18:53:40.659255 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:40.659816 30395 external_mini_cluster.cc:1428] Setting key f6297d39ca72c64956f036a9aa60ef23
I20260810 18:53:40.671474 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35: Bootstrap starting.
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:53:40.674621 786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:40.675789 786 log.cc:824] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:40.679538 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35: No bootstrap required, opened a new log
I20260810 18:53:40.681712 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.661629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57307 (local address 127.0.0.1:33989)
0810 18:53:40.662401 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.662415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.662464 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:53:40.663977 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.663997 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.664367 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.664743 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:40.667099 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.668376 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.669648 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.670293 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.670478 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.674630 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:40.674638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:40.674652 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:40.674698 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:40.677743 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:40.678437 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:40.678444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:40.678451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:40.678626 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:40.679004 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:40.679008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:40.679010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:40.679549 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:40.679804 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:40.680259 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 18:53:40.680586 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":339,"threads_started":1}
I20260810 18:53:40.685448 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:40.689209 786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } }
I20260810 18:53:40.689636 786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:40.689742 786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089363ee41db44c5a710209ef1f72b35, State: Initialized, Role: FOLLOWER
I20260810 18:53:40.690317 786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } }
I20260810 18:53:40.690655 786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:40.690841 786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:40.691023 786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:40.693874 786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } }
I20260810 18:53:40.694494 786 leader_election.cc:302] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089363ee41db44c5a710209ef1f72b35; no voters:
I20260810 18:53:40.695119 786 leader_election.cc:288] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:40.695453 803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:40.697693 786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:40.697923 803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [term 1 LEADER]: Becoming Leader. State: Replica: 089363ee41db44c5a710209ef1f72b35, State: Running, Role: LEADER
I20260810 18:53:40.698454 803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } }
I20260810 18:53:40.700445 803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089363ee41db44c5a710209ef1f72b35. Latest consensus state: current_term: 1 leader_uuid: "089363ee41db44c5a710209ef1f72b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } } }
I20260810 18:53:40.700768 803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.701447 804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089363ee41db44c5a710209ef1f72b35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089363ee41db44c5a710209ef1f72b35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33989 } } }
I20260810 18:53:40.701789 804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.705118 813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:40.707392 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.690553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42167 (local address 127.0.0.1:33989)
0810 18:53:40.690722 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.690728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.691216 (+ 488us) server_negotiation.cc:407] Connection header received
0810 18:53:40.691410 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.691414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.691470 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.691541 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:40.693251 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.693915 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.696184 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.696848 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.696956 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.700224 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:40.700232 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:40.700237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:40.700281 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:40.704017 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:40.704956 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:40.704963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:40.704967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:40.705126 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:40.705623 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:40.705628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:40.705631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:40.706085 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:40.706381 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:40.706801 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 18:53:40.707024 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 18:53:40.710119 813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:40.718267 813 catalog_manager.cc:1347] Generated new cluster ID: e43cf8c84b7a49a3b9e691fe13a239e0
I20260810 18:53:40.718386 813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:40.765872 813 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:40.767807 813 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:40.777222 813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089363ee41db44c5a710209ef1f72b35: Generated new TSK 0
I20260810 18:53:40.778226 813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:40.786764 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 664
I20260810 18:53:40.787739 778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:40.877774 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 664
2026-08-10T18:53:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2165 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[837](info): set up 2 sockets
Aug 10 18:53:40 dist-test-slave-6zbq krb5kdc[837](info): commencing operation
krb5kdc: starting...
W20260810 18:53:42.111383 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:42 dist-test-slave-6zbq krb5kdc[837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:42.413348 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:42.745033 861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:42.745303 861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:42.745445 861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:42.752576 861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:42.752682 861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:42.752756 861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:42.752847 861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:42.752907 861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:42.762681 861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:42.763919 861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:42.765192 861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:42.776471 868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:42.776532 867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:42.776562 871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:42.777370 861 server_base.cc:1034] running on GCE node
I20260810 18:53:42.778024 861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:42.779330 861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:42.780507 861 hybrid_clock.cc:648] HybridClock initialized: now 1786388022780464 us; error 42 us; skew 500 ppm
I20260810 18:53:42.780804 861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:42 dist-test-slave-6zbq krb5kdc[837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388022, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:42.787746 861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:42.789062 861 webserver.cc:460] Webserver started at http://127.0.0.1:36495/ using document root <none> and password file <none>
I20260810 18:53:42.789669 861 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:42.789775 861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:42.790015 861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:42.793185 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2"
format_stamp: "Formatted at 2026-08-10 18:53:42 on dist-test-slave-6zbq"
server_key: "5505c058790e7f5bca8541ca28c1b9be"
server_key_iv: "414ddcfee10a9d50d35ca2116b7ea042"
server_key_version: "encryptionkey@0"
I20260810 18:53:42.793923 861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2"
format_stamp: "Formatted at 2026-08-10 18:53:42 on dist-test-slave-6zbq"
server_key: "5505c058790e7f5bca8541ca28c1b9be"
server_key_iv: "414ddcfee10a9d50d35ca2116b7ea042"
server_key_version: "encryptionkey@0"
I20260810 18:53:42.798810 861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 18:53:42.802102 878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:42.803310 861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:42.803457 861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2"
format_stamp: "Formatted at 2026-08-10 18:53:42 on dist-test-slave-6zbq"
server_key: "5505c058790e7f5bca8541ca28c1b9be"
server_key_iv: "414ddcfee10a9d50d35ca2116b7ea042"
server_key_version: "encryptionkey@0"
I20260810 18:53:42.803638 861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:42.817734 861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:42.821334 861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:42.821571 861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:42.838083 861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41109
I20260810 18:53:42.838121 953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41109 every 8 connection(s)
I20260810 18:53:42.839321 861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:42.842741 955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:42.845805 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 861
I20260810 18:53:42.845988 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:42.846581 30395 external_mini_cluster.cc:1428] Setting key 7f2fea7253245571e0af6be002eb9394
I20260810 18:53:42.854183 955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2: Bootstrap starting.
I20260810 18:53:42.857524 955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:42.858677 955 log.cc:824] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:42.858968 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:42.848232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51881 (local address 127.0.0.1:41109)
0810 18:53:42.848943 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:42.848953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:42.848990 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:53:42.850612 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:42.850631 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:42.851010 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:42.851306 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:42.851858 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:42.853090 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:42.854907 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:42.855524 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:42.855666 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:42.856812 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:42.856819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:42.856831 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:42.857311 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:42.857381 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:42.857405 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:42.857498 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:42.857739 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 18:53:42.857999 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":329,"threads_started":1}
I20260810 18:53:42.861269 955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2: No bootstrap required, opened a new log
I20260810 18:53:42.862215 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:42.870689 955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260810 18:53:42.871078 955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:42.871178 955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea3d634ebe743ccae2a06fe0fb96aa2, State: Initialized, Role: FOLLOWER
I20260810 18:53:42.871802 955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260810 18:53:42.872047 955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:42.872182 955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:42.872344 955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:42.873412 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:42.867149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51487 (local address 127.0.0.1:41109)
0810 18:53:42.867322 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:42.867328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:42.867752 (+ 424us) server_negotiation.cc:407] Connection header received
0810 18:53:42.867898 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:42.867907 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:42.867986 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:42.868094 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:42.868503 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:42.869587 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:42.870983 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:42.871623 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:42.871744 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:42.872727 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:42.872733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:42.872737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:42.872757 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:42.872800 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:42.872821 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:42.872925 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:42.872996 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 18:53:42.873094 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:42.874969 955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260810 18:53:42.875386 955 leader_election.cc:302] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ea3d634ebe743ccae2a06fe0fb96aa2; no voters:
I20260810 18:53:42.875881 955 leader_election.cc:288] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:42.875999 970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:42.876679 970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea3d634ebe743ccae2a06fe0fb96aa2, State: Running, Role: LEADER
I20260810 18:53:42.877166 970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260810 18:53:42.877820 955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:42.879596 973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ea3d634ebe743ccae2a06fe0fb96aa2. Latest consensus state: current_term: 1 leader_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260810 18:53:42.879957 973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:42.880196 971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea3d634ebe743ccae2a06fe0fb96aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260810 18:53:42.880510 971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:42.882941 978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:42.889072 978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:42.896979 978 catalog_manager.cc:1347] Generated new cluster ID: 83634fefd7d94ff69b7ab94ab00bbb21
I20260810 18:53:42.897076 978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:42.930066 978 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:42.931335 978 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:42.936549 978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ea3d634ebe743ccae2a06fe0fb96aa2: Generated new TSK 0
I20260810 18:53:42.937235 978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:42.951380 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 861
W20260810 18:53:42.952385 949 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51881
W20260810 18:53:42.953014 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 861: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:42.953166 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 861
2026-08-10T18:53:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:43 dist-test-slave-6zbq krb5kdc[1008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:43 dist-test-slave-6zbq krb5kdc[1008](info): set up 2 sockets
Aug 10 18:53:43 dist-test-slave-6zbq krb5kdc[1008](info): commencing operation
krb5kdc: starting...
W20260810 18:53:44.173365 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.010s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:44 dist-test-slave-6zbq krb5kdc[1008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:44.441164 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:44.760828 1033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:44.761086 1033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:44.761190 1033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:44.768002 1033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:44.768095 1033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:44.768147 1033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:44.768205 1033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:44.768249 1033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:44.777457 1033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:44.778560 1033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:44.779733 1033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:44.790380 1042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.790444 1041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.790872 1044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:44.791275 1033 server_base.cc:1034] running on GCE node
I20260810 18:53:44.791924 1033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:44.793089 1033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:44.794286 1033 hybrid_clock.cc:648] HybridClock initialized: now 1786388024794244 us; error 30 us; skew 500 ppm
I20260810 18:53:44.794615 1033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:44 dist-test-slave-6zbq krb5kdc[1008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.797469 1033 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:44.798754 1033 webserver.cc:460] Webserver started at http://127.0.0.1:39433/ using document root <none> and password file <none>
I20260810 18:53:44.799314 1033 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:44.799407 1033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:44.799641 1033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:44.802306 1033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "148c6a5be9d040a983eb187567576ab2"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-6zbq"
server_key: "700bf070641ac569afa9a316259f6385"
server_key_iv: "4f2ebb452011c110894bde4824adc630"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.803036 1033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "148c6a5be9d040a983eb187567576ab2"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-6zbq"
server_key: "700bf070641ac569afa9a316259f6385"
server_key_iv: "4f2ebb452011c110894bde4824adc630"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.807617 1033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:44.810678 1052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.811827 1033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:44.811964 1033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "148c6a5be9d040a983eb187567576ab2"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-6zbq"
server_key: "700bf070641ac569afa9a316259f6385"
server_key_iv: "4f2ebb452011c110894bde4824adc630"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.812134 1033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:44.817402 1033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:44.820827 1033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:44.821101 1033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:44.837410 1033 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35589
I20260810 18:53:44.837467 1128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35589 every 8 connection(s)
I20260810 18:53:44.838644 1033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:44.841534 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1033
I20260810 18:53:44.841702 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:44.841969 1130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:44.842135 30395 external_mini_cluster.cc:1428] Setting key 5a21da5a4e30ef438583893c0fb549af
I20260810 18:53:44.850050 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.843704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50481 (local address 127.0.0.1:35589)
0810 18:53:44.844525 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.844539 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.844593 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:53:44.846582 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.846607 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.846989 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.847393 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.848313 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.848321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.848336 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:44.848357 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.848426 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:44.848465 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.848659 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.848782 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.848947 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":323,"threads_started":1}
I20260810 18:53:44.853142 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:44.855005 1130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2: Bootstrap starting.
I20260810 18:53:44.857941 1130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:44.859074 1130 log.cc:824] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:44.859315 1133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.857239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48641 (local address 127.0.0.1:35589)
0810 18:53:44.857396 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.857401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.857644 (+ 243us) server_negotiation.cc:407] Connection header received
0810 18:53:44.857753 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.857758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.857809 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.857912 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.858697 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.858702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.858706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:44.858723 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.858769 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:44.858787 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.858906 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.858949 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.858997 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:44.861945 1130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2: No bootstrap required, opened a new log
I20260810 18:53:44.869143 1130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260810 18:53:44.869460 1130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:44.869531 1130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148c6a5be9d040a983eb187567576ab2, State: Initialized, Role: FOLLOWER
I20260810 18:53:44.869987 1130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260810 18:53:44.870191 1130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:44.870287 1130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:44.870433 1130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:44.872403 1130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260810 18:53:44.872803 1130 leader_election.cc:302] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 148c6a5be9d040a983eb187567576ab2; no voters:
I20260810 18:53:44.873377 1130 leader_election.cc:288] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:44.873499 1141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:44.874159 1141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [term 1 LEADER]: Becoming Leader. State: Replica: 148c6a5be9d040a983eb187567576ab2, State: Running, Role: LEADER
I20260810 18:53:44.874706 1141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260810 18:53:44.875355 1130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:44.877247 1143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "148c6a5be9d040a983eb187567576ab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } } }
I20260810 18:53:44.877645 1143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:44.878793 1155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:44.878760 1146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148c6a5be9d040a983eb187567576ab2. Latest consensus state: current_term: 1 leader_uuid: "148c6a5be9d040a983eb187567576ab2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148c6a5be9d040a983eb187567576ab2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } } }
I20260810 18:53:44.879074 1146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:44.882887 1155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:44.890126 1155 catalog_manager.cc:1347] Generated new cluster ID: dae5ebdbe32648da861809ae979dafc5
I20260810 18:53:44.890221 1155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:44.920182 1155 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:44.920739 1155 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:44.928370 1155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 148c6a5be9d040a983eb187567576ab2: Generated new TSK 0
I20260810 18:53:44.929123 1155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:44.935931 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1033
W20260810 18:53:44.936987 1120 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50481
W20260810 18:53:44.937847 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1033: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:44.937983 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1033
2026-08-10T18:53:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:45 dist-test-slave-6zbq krb5kdc[1173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:45 dist-test-slave-6zbq krb5kdc[1173](info): set up 2 sockets
Aug 10 18:53:45 dist-test-slave-6zbq krb5kdc[1173](info): commencing operation
krb5kdc: starting...
W20260810 18:53:46.179557 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:46 dist-test-slave-6zbq krb5kdc[1173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:46.432958 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:46.748461 1202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:46.748695 1202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:46.748811 1202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:46.755591 1202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:46.755678 1202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:46.755748 1202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:46.755825 1202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:46.755880 1202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:46.765094 1202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:46.766086 1202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:46.767217 1202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:46.777773 1210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.777854 1208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.777837 1213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.778546 1202 server_base.cc:1034] running on GCE node
I20260810 18:53:46.779181 1202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:46.780360 1202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:46.781551 1202 hybrid_clock.cc:648] HybridClock initialized: now 1786388026781517 us; error 27 us; skew 500 ppm
I20260810 18:53:46.781864 1202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:46 dist-test-slave-6zbq krb5kdc[1173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:46.784778 1202 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:46.786018 1202 webserver.cc:460] Webserver started at http://127.0.0.1:34647/ using document root <none> and password file <none>
I20260810 18:53:46.786662 1202 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:46.786778 1202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:46.787043 1202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:46.789718 1202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "fb1e61a198be42cab2afac081ed15f95"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-6zbq"
server_key: "d9f7b2250091f5b5296138cde9870bb2"
server_key_iv: "e7e98c8566778f6e51367badf1b50696"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.790499 1202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "fb1e61a198be42cab2afac081ed15f95"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-6zbq"
server_key: "d9f7b2250091f5b5296138cde9870bb2"
server_key_iv: "e7e98c8566778f6e51367badf1b50696"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.795145 1202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:46.798156 1229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.799389 1202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:46.799542 1202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "fb1e61a198be42cab2afac081ed15f95"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-6zbq"
server_key: "d9f7b2250091f5b5296138cde9870bb2"
server_key_iv: "e7e98c8566778f6e51367badf1b50696"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.799705 1202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:46.815725 1202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:46.819870 1202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:46.820200 1202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:46.838380 1202 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46533
I20260810 18:53:46.838429 1318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46533 every 8 connection(s)
I20260810 18:53:46.839594 1202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:46.843000 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:46.844403 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1202
I20260810 18:53:46.844565 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:46.844990 30395 external_mini_cluster.cc:1428] Setting key f3dd980f2abbdf9f034b12e7c3ad2198
I20260810 18:53:46.856276 1319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95: Bootstrap starting.
Aug 10 18:53:46 dist-test-slave-6zbq krb5kdc[1173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:53:46.859088 1319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:46.860275 1319 log.cc:824] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:46.862969 1319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95: No bootstrap required, opened a new log
I20260810 18:53:46.866667 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.846754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:55731 (local address 127.0.0.1:46533)
0810 18:53:46.847545 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.847559 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.847607 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:53:46.849214 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.849249 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.849729 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.850133 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.851476 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.853416 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.854721 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.855688 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.855871 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.859620 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.859630 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.859646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.859686 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.862782 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.863509 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.863515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.863521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.863676 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.864095 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.864099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.864101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.864622 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.864939 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.865427 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.865716 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":298,"threads_started":1}
I20260810 18:53:46.870033 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:46.870595 1319 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260810 18:53:46.870898 1319 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:46.870978 1319 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb1e61a198be42cab2afac081ed15f95, State: Initialized, Role: FOLLOWER
I20260810 18:53:46.871452 1319 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260810 18:53:46.871672 1319 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:46.871791 1319 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:46.871933 1319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:46.874053 1319 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260810 18:53:46.874634 1319 leader_election.cc:302] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb1e61a198be42cab2afac081ed15f95; no voters:
I20260810 18:53:46.875217 1319 leader_election.cc:288] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:46.875365 1335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:46.875985 1335 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [term 1 LEADER]: Becoming Leader. State: Replica: fb1e61a198be42cab2afac081ed15f95, State: Running, Role: LEADER
I20260810 18:53:46.876595 1335 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } }
I20260810 18:53:46.878763 1319 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:46.879385 1340 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb1e61a198be42cab2afac081ed15f95. Latest consensus state: current_term: 1 leader_uuid: "fb1e61a198be42cab2afac081ed15f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } } }
I20260810 18:53:46.879740 1340 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:46.884804 1341 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb1e61a198be42cab2afac081ed15f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1e61a198be42cab2afac081ed15f95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46533 } } }
I20260810 18:53:46.885650 1341 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:46.888824 1345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:46.891783 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.874374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:44499 (local address 127.0.0.1:46533)
0810 18:53:46.874536 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.874541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.874893 (+ 352us) server_negotiation.cc:407] Connection header received
0810 18:53:46.875027 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.875033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.875113 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.875235 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.877071 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.878244 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.879958 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.880648 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.880772 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.884012 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.884020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.884025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.884064 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.887636 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.888333 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.888342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.888346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.888505 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.889084 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.889089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.889092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.889500 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.889778 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.891032 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.891313 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:53:46.894742 1345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:46.903131 1345 catalog_manager.cc:1347] Generated new cluster ID: a608e287ccac4947a35ddad6abbfcf45
I20260810 18:53:46.903234 1345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:46.913204 1345 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:46.914319 1345 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:46.923847 1345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb1e61a198be42cab2afac081ed15f95: Generated new TSK 0
I20260810 18:53:46.924808 1345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:46.937002 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1202
I20260810 18:53:46.938052 1309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:47.026057 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1202
W20260810 18:53:47.046742 1171 connection.cc:535] client connection to 127.0.0.1:46533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46533): Connection reset by peer (error 104)
2026-08-10T18:53:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:47 dist-test-slave-6zbq krb5kdc[1374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:47 dist-test-slave-6zbq krb5kdc[1374](info): set up 2 sockets
Aug 10 18:53:47 dist-test-slave-6zbq krb5kdc[1374](info): commencing operation
krb5kdc: starting...
W20260810 18:53:48.264334 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:48 dist-test-slave-6zbq krb5kdc[1374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:48.510638 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39397
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:48.828359 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:48.828612 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:48.828707 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:48.835476 1401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:48.835563 1401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:48.835614 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:48.835672 1401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:48.835714 1401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:48.844736 1401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:48.845788 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:48.846944 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:48.857533 1417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.857584 1419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.857607 1421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:48.858251 1401 server_base.cc:1034] running on GCE node
I20260810 18:53:48.858955 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:48.860162 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:48.861364 1401 hybrid_clock.cc:648] HybridClock initialized: now 1786388028861323 us; error 36 us; skew 500 ppm
I20260810 18:53:48.861668 1401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:48 dist-test-slave-6zbq krb5kdc[1374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:48.868086 1401 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:48.869400 1401 webserver.cc:460] Webserver started at http://127.0.0.1:40641/ using document root <none> and password file <none>
I20260810 18:53:48.869959 1401 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:48.870065 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:48.870368 1401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:48.873021 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "a561cf754d674e0398914c1e4f580050"
format_stamp: "Formatted at 2026-08-10 18:53:48 on dist-test-slave-6zbq"
server_key: "2e4ae94a12645a053dc2dfba59ad15db"
server_key_iv: "068611f16700a2c9a5cdf436b3054dcc"
server_key_version: "encryptionkey@0"
I20260810 18:53:48.873713 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "a561cf754d674e0398914c1e4f580050"
format_stamp: "Formatted at 2026-08-10 18:53:48 on dist-test-slave-6zbq"
server_key: "2e4ae94a12645a053dc2dfba59ad15db"
server_key_iv: "068611f16700a2c9a5cdf436b3054dcc"
server_key_version: "encryptionkey@0"
I20260810 18:53:48.878221 1401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 18:53:48.881305 1433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.882441 1401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:48.882591 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "a561cf754d674e0398914c1e4f580050"
format_stamp: "Formatted at 2026-08-10 18:53:48 on dist-test-slave-6zbq"
server_key: "2e4ae94a12645a053dc2dfba59ad15db"
server_key_iv: "068611f16700a2c9a5cdf436b3054dcc"
server_key_version: "encryptionkey@0"
I20260810 18:53:48.882748 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:48.894133 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:48.897541 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:48.897778 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:48.914206 1401 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39397
I20260810 18:53:48.914243 1524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39397 every 8 connection(s)
I20260810 18:53:48.915390 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:48.918639 1525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:48.920868 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1401
I20260810 18:53:48.921030 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:48.921438 30395 external_mini_cluster.cc:1428] Setting key 0460c360384e702f17e8f59073873ff1
I20260810 18:53:48.930985 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050: Bootstrap starting.
I20260810 18:53:48.934585 1525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:48.935554 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.923080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:54733 (local address 127.0.0.1:39397)
0810 18:53:48.923883 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.923898 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.923952 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:53:48.926187 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.926215 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.926737 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.927162 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.927643 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.929538 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.930844 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.931819 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.931997 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.933004 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.933015 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.933031 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:48.933683 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.933791 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:48.933825 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.933975 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.934134 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.934302 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":318,"threads_started":1}
I20260810 18:53:48.935901 1525 log.cc:824] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:48.938874 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050: No bootstrap required, opened a new log
I20260810 18:53:48.939289 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:48.946297 1525 raft_consensus.cc:348] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } }
I20260810 18:53:48.946656 1525 raft_consensus.cc:374] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:48.946767 1525 raft_consensus.cc:729] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a561cf754d674e0398914c1e4f580050, State: Initialized, Role: FOLLOWER
I20260810 18:53:48.947283 1525 consensus_queue.cc:260] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } }
I20260810 18:53:48.947520 1525 raft_consensus.cc:388] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:48.947651 1525 raft_consensus.cc:482] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:48.947808 1525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:48.949080 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.943340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35987 (local address 127.0.0.1:39397)
0810 18:53:48.943496 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.943502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.943859 (+ 357us) server_negotiation.cc:407] Connection header received
0810 18:53:48.944025 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.944029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.944083 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.944165 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.944620 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.945372 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.946681 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.947240 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.947351 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.948377 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.948385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.948388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:48.948403 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.948458 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:48.948473 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.948576 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.948773 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.948821 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:53:48.950382 1525 raft_consensus.cc:504] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } }
I20260810 18:53:48.950851 1525 leader_election.cc:302] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a561cf754d674e0398914c1e4f580050; no voters:
I20260810 18:53:48.951341 1525 leader_election.cc:288] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:48.951435 1545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:48.952107 1545 raft_consensus.cc:686] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [term 1 LEADER]: Becoming Leader. State: Replica: a561cf754d674e0398914c1e4f580050, State: Running, Role: LEADER
I20260810 18:53:48.952670 1545 consensus_queue.cc:237] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } }
I20260810 18:53:48.953215 1525 sys_catalog.cc:564] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:48.955229 1551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a561cf754d674e0398914c1e4f580050. Latest consensus state: current_term: 1 leader_uuid: "a561cf754d674e0398914c1e4f580050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } } }
I20260810 18:53:48.955518 1551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.956045 1548 sys_catalog.cc:455] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a561cf754d674e0398914c1e4f580050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561cf754d674e0398914c1e4f580050" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39397 } } }
I20260810 18:53:48.956336 1548 sys_catalog.cc:458] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.957108 1559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:48.961855 1559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:48.970409 1559 catalog_manager.cc:1347] Generated new cluster ID: 0b4adf3cb25d45ffad52c1a080f8635e
I20260810 18:53:48.970506 1559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:48.985631 1559 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:48.987308 1559 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:49.009300 1559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a561cf754d674e0398914c1e4f580050: Generated new TSK 0
I20260810 18:53:49.010257 1559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:49.021489 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1401
W20260810 18:53:49.022500 1517 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:54733
W20260810 18:53:49.023190 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1401: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:49.023345 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1401
W20260810 18:53:49.048818 1371 connection.cc:535] client connection to 127.0.0.1:39397 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39397): Connection reset by peer (error 104)
2026-08-10T18:53:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:49 dist-test-slave-6zbq krb5kdc[1582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:49 dist-test-slave-6zbq krb5kdc[1582](info): set up 2 sockets
Aug 10 18:53:49 dist-test-slave-6zbq krb5kdc[1582](info): commencing operation
krb5kdc: starting...
W20260810 18:53:50.242300 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:50 dist-test-slave-6zbq krb5kdc[1582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:50.494990 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:50.813220 1607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:50.813472 1607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:50.813594 1607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:50.820516 1607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:50.820611 1607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:50.820685 1607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:50.820767 1607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:50.820827 1607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:50.830204 1607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:50.831351 1607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:50.832543 1607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:50.843417 1623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:50.843441 1621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:50.843665 1617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:50.843945 1607 server_base.cc:1034] running on GCE node
I20260810 18:53:50.844775 1607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:50.845992 1607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:50.847182 1607 hybrid_clock.cc:648] HybridClock initialized: now 1786388030847130 us; error 47 us; skew 500 ppm
I20260810 18:53:50.847491 1607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:50 dist-test-slave-6zbq krb5kdc[1582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:50.850394 1607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:50.851658 1607 webserver.cc:460] Webserver started at http://127.0.0.1:36147/ using document root <none> and password file <none>
I20260810 18:53:50.852214 1607 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:50.852320 1607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:50.852553 1607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:50.855283 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "333ff2e18bb84d6fb9f5304d21f70c62"
format_stamp: "Formatted at 2026-08-10 18:53:50 on dist-test-slave-6zbq"
server_key: "0c1c465c6f7827f0466951e65f05554b"
server_key_iv: "27090f994e479971633146089a8566f5"
server_key_version: "encryptionkey@0"
I20260810 18:53:50.855971 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "333ff2e18bb84d6fb9f5304d21f70c62"
format_stamp: "Formatted at 2026-08-10 18:53:50 on dist-test-slave-6zbq"
server_key: "0c1c465c6f7827f0466951e65f05554b"
server_key_iv: "27090f994e479971633146089a8566f5"
server_key_version: "encryptionkey@0"
I20260810 18:53:50.860585 1607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:50.863735 1631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:50.864866 1607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:50.865005 1607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "333ff2e18bb84d6fb9f5304d21f70c62"
format_stamp: "Formatted at 2026-08-10 18:53:50 on dist-test-slave-6zbq"
server_key: "0c1c465c6f7827f0466951e65f05554b"
server_key_iv: "27090f994e479971633146089a8566f5"
server_key_version: "encryptionkey@0"
I20260810 18:53:50.865157 1607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:50.870186 1607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:50.874063 1607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:50.874300 1607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:50.890133 1607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45135
I20260810 18:53:50.890199 1719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45135 every 8 connection(s)
I20260810 18:53:50.891320 1607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:50.894639 1721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:50.895059 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1607
I20260810 18:53:50.895262 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:50.895798 30395 external_mini_cluster.cc:1428] Setting key 26366c7645520dda6c437bcc752f7f61
I20260810 18:53:50.903386 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.897416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:50165 (local address 127.0.0.1:45135)
0810 18:53:50.898153 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:50.898163 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:50.898203 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:53:50.900418 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:50.900438 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:50.900744 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:50.901040 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:50.901971 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:50.901977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:50.901989 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:50.902002 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:50.902056 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:50.902077 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:50.902226 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:50.902391 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 18:53:50.902575 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":338,"threads_started":1}
I20260810 18:53:50.906899 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:50.907330 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: Bootstrap starting.
I20260810 18:53:50.910878 1721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:50.912072 1721 log.cc:824] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:50.914165 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.911868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:53705 (local address 127.0.0.1:45135)
0810 18:53:50.912026 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:50.912031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:50.912464 (+ 433us) server_negotiation.cc:407] Connection header received
0810 18:53:50.912605 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:50.912610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:50.912671 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:50.912749 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:50.913620 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:50.913626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:50.913630 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:50.913648 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:50.913692 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:50.913712 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:50.913823 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:50.913864 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 18:53:50.913913 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:53:50.915045 1721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: No bootstrap required, opened a new log
I20260810 18:53:50.921854 1721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260810 18:53:50.922159 1721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:50.922230 1721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 333ff2e18bb84d6fb9f5304d21f70c62, State: Initialized, Role: FOLLOWER
I20260810 18:53:50.922727 1721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260810 18:53:50.922928 1721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:50.923024 1721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:50.923146 1721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:50.925047 1721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260810 18:53:50.925454 1721 leader_election.cc:302] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 333ff2e18bb84d6fb9f5304d21f70c62; no voters:
I20260810 18:53:50.925992 1721 leader_election.cc:288] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:50.926138 1739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:50.926820 1739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [term 1 LEADER]: Becoming Leader. State: Replica: 333ff2e18bb84d6fb9f5304d21f70c62, State: Running, Role: LEADER
I20260810 18:53:50.927330 1739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } }
I20260810 18:53:50.927862 1721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:50.929890 1740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } } }
I20260810 18:53:50.930395 1740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:50.930248 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 333ff2e18bb84d6fb9f5304d21f70c62. Latest consensus state: current_term: 1 leader_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "333ff2e18bb84d6fb9f5304d21f70c62" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45135 } } }
I20260810 18:53:50.930614 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62 [sys.catalog]: This master's current role is: LEADER
W20260810 18:53:50.939221 1767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:50.939350 1767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:50.941766 1747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:50.943400 1747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:50.950817 1747 catalog_manager.cc:1347] Generated new cluster ID: 30efd42435ea4cbe9208d5d35623f25e
I20260810 18:53:50.950901 1747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:50.973006 1747 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:50.973346 1747 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:50.995527 1747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 333ff2e18bb84d6fb9f5304d21f70c62: Generated new TSK 0
I20260810 18:53:50.996232 1747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:51.004905 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1607
W20260810 18:53:51.006330 1708 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:50165
W20260810 18:53:51.006973 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1607: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:51.007092 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1607
2026-08-10T18:53:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1984 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:51 dist-test-slave-6zbq krb5kdc[1775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:51 dist-test-slave-6zbq krb5kdc[1775](info): set up 2 sockets
Aug 10 18:53:51 dist-test-slave-6zbq krb5kdc[1775](info): commencing operation
krb5kdc: starting...
W20260810 18:53:52.242252 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:52 dist-test-slave-6zbq krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:52.509073 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:52.823022 1798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:52.823269 1798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:52.823364 1798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:52.833211 1798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:52.833333 1798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:52.833402 1798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:52.833468 1798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:52.833511 1798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:52.842744 1798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:52.843742 1798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:52.844857 1798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:52.855454 1808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:52.855414 1813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:52.855587 1809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:52.856330 1798 server_base.cc:1034] running on GCE node
I20260810 18:53:52.857035 1798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:52.858246 1798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:52.859440 1798 hybrid_clock.cc:648] HybridClock initialized: now 1786388032859402 us; error 57 us; skew 500 ppm
I20260810 18:53:52.859750 1798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:52 dist-test-slave-6zbq krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:52.862619 1798 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:52.863889 1798 webserver.cc:460] Webserver started at http://127.0.0.1:37519/ using document root <none> and password file <none>
I20260810 18:53:52.864483 1798 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:52.864602 1798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:52.864869 1798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:52.867621 1798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "4c8f80c359e841d2a466e00414f8c091"
format_stamp: "Formatted at 2026-08-10 18:53:52 on dist-test-slave-6zbq"
server_key: "e2e6ccd9f47ab3ea33685b4392ff8182"
server_key_iv: "c80376616fc2984c37da13c2648ae267"
server_key_version: "encryptionkey@0"
I20260810 18:53:52.868347 1798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "4c8f80c359e841d2a466e00414f8c091"
format_stamp: "Formatted at 2026-08-10 18:53:52 on dist-test-slave-6zbq"
server_key: "e2e6ccd9f47ab3ea33685b4392ff8182"
server_key_iv: "c80376616fc2984c37da13c2648ae267"
server_key_version: "encryptionkey@0"
I20260810 18:53:52.872916 1798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:52.875979 1823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:52.877063 1798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:52.877238 1798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "4c8f80c359e841d2a466e00414f8c091"
format_stamp: "Formatted at 2026-08-10 18:53:52 on dist-test-slave-6zbq"
server_key: "e2e6ccd9f47ab3ea33685b4392ff8182"
server_key_iv: "c80376616fc2984c37da13c2648ae267"
server_key_version: "encryptionkey@0"
I20260810 18:53:52.877398 1798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:52.896203 1798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:52.900070 1798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:52.900313 1798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:52.916530 1798 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42069
I20260810 18:53:52.916564 1908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42069 every 8 connection(s)
I20260810 18:53:52.917728 1798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:52.920192 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1798
I20260810 18:53:52.920362 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:52.920783 30395 external_mini_cluster.cc:1428] Setting key c8cce6f3de5099c019427169b8d5aba8
I20260810 18:53:52.921180 1909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:52.934262 1909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091: Bootstrap starting.
Aug 10 18:53:52 dist-test-slave-6zbq krb5kdc[1775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:53:52.937536 1909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:52.938845 1909 log.cc:824] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:52.941413 1909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091: No bootstrap required, opened a new log
I20260810 18:53:52.943517 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.922377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:37469 (local address 127.0.0.1:42069)
0810 18:53:52.923138 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.923151 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.923201 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:53:52.925557 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.925583 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.926046 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.926469 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:52.927701 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.929641 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.930919 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.931937 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.932124 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.935963 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:52.935974 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:52.935992 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:52.936032 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:52.939698 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:52.940407 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:52.940413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:52.940419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:52.940571 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:52.940993 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:52.940996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:52.940998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:52.941496 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:52.941807 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.942283 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.942592 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":299,"threads_started":1}
I20260810 18:53:52.947170 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:52.949416 1909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260810 18:53:52.949716 1909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:52.949811 1909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c8f80c359e841d2a466e00414f8c091, State: Initialized, Role: FOLLOWER
I20260810 18:53:52.950318 1909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260810 18:53:52.950564 1909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:52.950686 1909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:52.950836 1909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:52.953217 1909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260810 18:53:52.953739 1909 leader_election.cc:302] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c8f80c359e841d2a466e00414f8c091; no voters:
I20260810 18:53:52.954368 1909 leader_election.cc:288] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:52.954452 1929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:52.956130 1909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:52.957556 1929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [term 1 LEADER]: Becoming Leader. State: Replica: 4c8f80c359e841d2a466e00414f8c091, State: Running, Role: LEADER
I20260810 18:53:52.958210 1929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260810 18:53:52.960697 1931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c8f80c359e841d2a466e00414f8c091. Latest consensus state: current_term: 1 leader_uuid: "4c8f80c359e841d2a466e00414f8c091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260810 18:53:52.961062 1931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:52.961623 1930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c8f80c359e841d2a466e00414f8c091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c8f80c359e841d2a466e00414f8c091" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260810 18:53:52.961903 1930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:52.967935 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.951372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:40031 (local address 127.0.0.1:42069)
0810 18:53:52.951534 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.951540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.951942 (+ 402us) server_negotiation.cc:407] Connection header received
0810 18:53:52.952045 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.952050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.952126 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.952227 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:52.953463 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.954577 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.956354 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.957055 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.957149 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.960398 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:52.960405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:52.960411 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:52.960449 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:52.964176 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:52.965188 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:52.965194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:52.965198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:52.965362 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:52.965917 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:52.965923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:52.965926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:52.966384 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:52.966654 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.967363 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.967547 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:53:52.969156 1950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:52.973351 1950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:52.981529 1950 catalog_manager.cc:1347] Generated new cluster ID: 8a1df5cd76cd417aba5a1b4b04481ea4
I20260810 18:53:52.981689 1950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:53.017584 1950 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:53.018793 1950 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:53.032222 1950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c8f80c359e841d2a466e00414f8c091: Generated new TSK 0
I20260810 18:53:53.032900 1950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:53.043568 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1798
I20260810 18:53:53.044615 1904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:53.135885 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1798
W20260810 18:53:53.157320 1772 connection.cc:535] client connection to 127.0.0.1:42069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42069): Connection reset by peer (error 104)
2026-08-10T18:53:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2128 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:53 dist-test-slave-6zbq krb5kdc[1974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:53 dist-test-slave-6zbq krb5kdc[1974](info): set up 2 sockets
Aug 10 18:53:53 dist-test-slave-6zbq krb5kdc[1974](info): commencing operation
krb5kdc: starting...
W20260810 18:53:54.375289 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:54 dist-test-slave-6zbq krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:54.629308 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:54.940372 1997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:54.940608 1997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:54.940730 1997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:54.947896 1997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:54.947989 1997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:54.948060 1997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:54.948148 1997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:54.948206 1997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:54.957234 1997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:54.958259 1997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:54.959416 1997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:54.969964 2005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.970038 2006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.970073 2009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:54.970821 1997 server_base.cc:1034] running on GCE node
I20260810 18:53:54.971474 1997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:54.972641 1997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:54.973838 1997 hybrid_clock.cc:648] HybridClock initialized: now 1786388034973792 us; error 38 us; skew 500 ppm
I20260810 18:53:54.974131 1997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:54 dist-test-slave-6zbq krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:54.976949 1997 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:54.978183 1997 webserver.cc:460] Webserver started at http://127.0.0.1:40859/ using document root <none> and password file <none>
I20260810 18:53:54.978796 1997 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:54.978893 1997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:54.979120 1997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:54.981789 1997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "54a6a859f70b42b98ac7dc051458e352"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-6zbq"
server_key: "85dad4f75cf1035cf26dc6e639240a2d"
server_key_iv: "c989122fad5be9d116061fc0567304e6"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.982501 1997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "54a6a859f70b42b98ac7dc051458e352"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-6zbq"
server_key: "85dad4f75cf1035cf26dc6e639240a2d"
server_key_iv: "c989122fad5be9d116061fc0567304e6"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.986960 1997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:53:54.990203 2020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.991357 1997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:54.991482 1997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "54a6a859f70b42b98ac7dc051458e352"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-6zbq"
server_key: "85dad4f75cf1035cf26dc6e639240a2d"
server_key_iv: "c989122fad5be9d116061fc0567304e6"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.991629 1997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:55.004034 1997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:55.007398 1997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:55.007606 1997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:55.023618 1997 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33525
I20260810 18:53:55.023671 2097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33525 every 8 connection(s)
I20260810 18:53:55.024755 1997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:55.027941 2099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.030931 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 1997
I20260810 18:53:55.031117 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:55.031658 30395 external_mini_cluster.cc:1428] Setting key aff0fedd76db2976d847eccc130e2007
I20260810 18:53:55.038812 2099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352: Bootstrap starting.
I20260810 18:53:55.041604 2099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.042644 2099 log.cc:824] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:55.043913 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.033420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:52431 (local address 127.0.0.1:33525)
0810 18:53:55.034069 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.034080 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.034116 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:53:55.035717 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.035736 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.036093 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.036428 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:55.036939 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.038164 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.039557 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.040212 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.040387 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.041533 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:55.041543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:55.041557 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:55.042084 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:55.042158 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:55.042182 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:55.042313 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.042622 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.042764 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":278,"threads_started":1}
I20260810 18:53:55.045300 2099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352: No bootstrap required, opened a new log
I20260810 18:53:55.047207 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:55.053084 2099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } }
I20260810 18:53:55.053445 2099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.053546 2099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54a6a859f70b42b98ac7dc051458e352, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.054147 2099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } }
I20260810 18:53:55.054441 2099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:55.054567 2099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:55.054733 2099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.057241 2105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.051028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:33123 (local address 127.0.0.1:33525)
0810 18:53:55.051198 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.051205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.051668 (+ 463us) server_negotiation.cc:407] Connection header received
0810 18:53:55.051831 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.051834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.051890 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.051965 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:55.052535 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.053228 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.054963 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.055427 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.055523 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.056568 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:55.056572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:55.056575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:55.056590 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:55.056653 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:55.056668 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:55.056745 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.056917 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.056953 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:53:55.057482 2099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } }
I20260810 18:53:55.057979 2099 leader_election.cc:302] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54a6a859f70b42b98ac7dc051458e352; no voters:
I20260810 18:53:55.058521 2099 leader_election.cc:288] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:55.058640 2119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:55.059238 2119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [term 1 LEADER]: Becoming Leader. State: Replica: 54a6a859f70b42b98ac7dc051458e352, State: Running, Role: LEADER
I20260810 18:53:55.059756 2119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } }
I20260810 18:53:55.060432 2099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:55.062496 2121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54a6a859f70b42b98ac7dc051458e352. Latest consensus state: current_term: 1 leader_uuid: "54a6a859f70b42b98ac7dc051458e352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } } }
I20260810 18:53:55.062522 2120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54a6a859f70b42b98ac7dc051458e352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a6a859f70b42b98ac7dc051458e352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33525 } } }
I20260810 18:53:55.062963 2120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:55.062963 2121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:55.068856 2127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:55.073750 2127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:55.082245 2127 catalog_manager.cc:1347] Generated new cluster ID: b626ce51cdab439db06ba62e1fbfeb9f
I20260810 18:53:55.082363 2127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:55.099400 2127 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:55.100994 2127 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:55.113682 2127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54a6a859f70b42b98ac7dc051458e352: Generated new TSK 0
I20260810 18:53:55.114533 2127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:55.121050 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1997
W20260810 18:53:55.122017 2091 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:52431
W20260810 18:53:55.122771 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 1997: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:55.122900 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 1997
W20260810 18:53:55.141623 1970 connection.cc:535] client connection to 127.0.0.1:33525 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33525): Connection reset by peer (error 104)
2026-08-10T18:53:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:55 dist-test-slave-6zbq krb5kdc[2152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:55 dist-test-slave-6zbq krb5kdc[2152](info): set up 2 sockets
Aug 10 18:53:55 dist-test-slave-6zbq krb5kdc[2152](info): commencing operation
krb5kdc: starting...
W20260810 18:53:56.342149 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:56 dist-test-slave-6zbq krb5kdc[2152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:53:56.613075 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:56.934798 2169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:56.935076 2169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:56.935210 2169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:56.942219 2169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:56.942350 2169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:56.942432 2169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:56.942515 2169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:56.942575 2169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:56.951987 2169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:56.953109 2169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:56.954335 2169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:56.965126 2180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:56.965163 2177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:56.965085 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:56.965898 2169 server_base.cc:1034] running on GCE node
I20260810 18:53:56.966637 2169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:56.967844 2169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:56.969043 2169 hybrid_clock.cc:648] HybridClock initialized: now 1786388036969008 us; error 23 us; skew 500 ppm
I20260810 18:53:56.969347 2169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:56 dist-test-slave-6zbq krb5kdc[2152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:56.972121 2169 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:53:56.973416 2169 webserver.cc:460] Webserver started at http://127.0.0.1:35341/ using document root <none> and password file <none>
I20260810 18:53:56.974005 2169 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:56.974118 2169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:56.974445 2169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:56.977176 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "e18d8462cc2344ba86946416b3d8ce1f"
format_stamp: "Formatted at 2026-08-10 18:53:56 on dist-test-slave-6zbq"
server_key: "32982f4fee4e182b9f4d87e06e644e31"
server_key_iv: "12033622c06166a7b1254f3f8c87bebf"
server_key_version: "encryptionkey@0"
I20260810 18:53:56.977891 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "e18d8462cc2344ba86946416b3d8ce1f"
format_stamp: "Formatted at 2026-08-10 18:53:56 on dist-test-slave-6zbq"
server_key: "32982f4fee4e182b9f4d87e06e644e31"
server_key_iv: "12033622c06166a7b1254f3f8c87bebf"
server_key_version: "encryptionkey@0"
I20260810 18:53:56.982623 2169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260810 18:53:56.985762 2197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:56.987032 2169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:56.987203 2169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "e18d8462cc2344ba86946416b3d8ce1f"
format_stamp: "Formatted at 2026-08-10 18:53:56 on dist-test-slave-6zbq"
server_key: "32982f4fee4e182b9f4d87e06e644e31"
server_key_iv: "12033622c06166a7b1254f3f8c87bebf"
server_key_version: "encryptionkey@0"
I20260810 18:53:56.987380 2169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:56.992657 2169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:56.996409 2169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:56.996658 2169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:57.013350 2169 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36673
I20260810 18:53:57.013406 2282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36673 every 8 connection(s)
I20260810 18:53:57.014706 2169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:57.017401 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2169
I20260810 18:53:57.017561 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:57.017997 30395 external_mini_cluster.cc:1428] Setting key 18b20565c4643201b567adca444e641b
I20260810 18:53:57.018323 2285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:57.025791 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:57.019617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:45771 (local address 127.0.0.1:36673)
0810 18:53:57.020374 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:57.020384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:57.020423 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:53:57.022708 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:57.022729 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:57.023047 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:57.023354 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:57.024356 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:57.024362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:57.024378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:57.024391 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:57.024456 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:57.024476 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:57.024650 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:57.024751 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 18:53:57.024930 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":322,"threads_started":1}
I20260810 18:53:57.029062 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:57.031157 2285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f: Bootstrap starting.
I20260810 18:53:57.034684 2285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:57.035784 2285 log.cc:824] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:57.036237 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:57.033847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35637 (local address 127.0.0.1:36673)
0810 18:53:57.034008 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:57.034014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:57.034449 (+ 435us) server_negotiation.cc:407] Connection header received
0810 18:53:57.034582 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:57.034586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:57.034653 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:57.034765 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:57.035571 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:57.035577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:57.035581 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:57.035603 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:57.035655 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:57.035675 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:57.035794 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:57.035919 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 18:53:57.035976 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:57.038546 2285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f: No bootstrap required, opened a new log
I20260810 18:53:57.045526 2285 raft_consensus.cc:348] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } }
I20260810 18:53:57.045835 2285 raft_consensus.cc:374] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:57.045924 2285 raft_consensus.cc:729] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18d8462cc2344ba86946416b3d8ce1f, State: Initialized, Role: FOLLOWER
I20260810 18:53:57.046437 2285 consensus_queue.cc:260] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } }
I20260810 18:53:57.046643 2285 raft_consensus.cc:388] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:57.046761 2285 raft_consensus.cc:482] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:57.046900 2285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:57.048873 2285 raft_consensus.cc:504] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } }
I20260810 18:53:57.049312 2285 leader_election.cc:302] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e18d8462cc2344ba86946416b3d8ce1f; no voters:
I20260810 18:53:57.049820 2285 leader_election.cc:288] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:57.049978 2305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:57.051141 2305 raft_consensus.cc:686] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [term 1 LEADER]: Becoming Leader. State: Replica: e18d8462cc2344ba86946416b3d8ce1f, State: Running, Role: LEADER
I20260810 18:53:57.051597 2305 consensus_queue.cc:237] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } }
I20260810 18:53:57.051858 2285 sys_catalog.cc:564] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:57.054091 2308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e18d8462cc2344ba86946416b3d8ce1f. Latest consensus state: current_term: 1 leader_uuid: "e18d8462cc2344ba86946416b3d8ce1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } } }
I20260810 18:53:57.054688 2308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:57.054095 2306 sys_catalog.cc:455] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e18d8462cc2344ba86946416b3d8ce1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e18d8462cc2344ba86946416b3d8ce1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36673 } } }
I20260810 18:53:57.055056 2306 sys_catalog.cc:458] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:57.056305 2318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:57.060765 2318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:57.068953 2318 catalog_manager.cc:1347] Generated new cluster ID: 344ebda7d37b420dab2655809a8753ab
I20260810 18:53:57.069067 2318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:57.090310 2318 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:57.090809 2318 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:57.111781 2318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e18d8462cc2344ba86946416b3d8ce1f: Generated new TSK 0
I20260810 18:53:57.112862 2318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:57.121131 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2169
W20260810 18:53:57.122709 2272 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.181:45771
W20260810 18:53:57.123340 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2169: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:53:57.123486 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2169
2026-08-10T18:53:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2005 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:57 dist-test-slave-6zbq krb5kdc[2347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:57 dist-test-slave-6zbq krb5kdc[2347](info): set up 2 sockets
Aug 10 18:53:57 dist-test-slave-6zbq krb5kdc[2347](info): commencing operation
krb5kdc: starting...
W20260810 18:53:58.364408 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:53:58 dist-test-slave-6zbq krb5kdc[2347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:53:58.639313 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35995
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:35995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:58.958549 2364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:58.958817 2364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:58.958948 2364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:58.965790 2364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:58.965896 2364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:58.965973 2364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:58.966054 2364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:58.966114 2364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:58.975555 2364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:35995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:35995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:53:58.976667 2364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:58.977852 2364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:58.988626 2380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:58.988574 2374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:58.988595 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:58.989285 2364 server_base.cc:1034] running on GCE node
I20260810 18:53:58.989954 2364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:58.991209 2364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:58.992406 2364 hybrid_clock.cc:648] HybridClock initialized: now 1786388038992365 us; error 32 us; skew 500 ppm
I20260810 18:53:58.992713 2364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:53:58 dist-test-slave-6zbq krb5kdc[2347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388038, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:58.995568 2364 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:53:58.996865 2364 webserver.cc:460] Webserver started at http://127.29.174.254:45119/ using document root <none> and password file <none>
I20260810 18:53:58.997431 2364 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:58.997530 2364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:58.997777 2364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:59.000706 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "7ce5a359f62b47868f32a891cdb377ec"
format_stamp: "Formatted at 2026-08-10 18:53:58 on dist-test-slave-6zbq"
server_key: "67572afc6bf40de9f2c6bcbea49b1b33"
server_key_iv: "2f759d40ddd50c99c520115659530d17"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.001416 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "7ce5a359f62b47868f32a891cdb377ec"
format_stamp: "Formatted at 2026-08-10 18:53:58 on dist-test-slave-6zbq"
server_key: "67572afc6bf40de9f2c6bcbea49b1b33"
server_key_iv: "2f759d40ddd50c99c520115659530d17"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.006246 2364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 18:53:59.009419 2388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.010632 2364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:59.010780 2364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "7ce5a359f62b47868f32a891cdb377ec"
format_stamp: "Formatted at 2026-08-10 18:53:58 on dist-test-slave-6zbq"
server_key: "67572afc6bf40de9f2c6bcbea49b1b33"
server_key_iv: "2f759d40ddd50c99c520115659530d17"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.010933 2364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:59.029227 2364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:59.032779 2364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:59.032999 2364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:59.050209 2364 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:35995
I20260810 18:53:59.050246 2472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:35995 every 8 connection(s)
I20260810 18:53:59.051424 2364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:59.054847 2474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:59.055106 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2364
I20260810 18:53:59.055286 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:53:59.055830 30395 external_mini_cluster.cc:1428] Setting key 4d7d00d641de27c3d8ec96948eb13119
I20260810 18:53:59.068225 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec: Bootstrap starting.
Aug 10 18:53:59 dist-test-slave-6zbq krb5kdc[2347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:53:59.071017 2474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:59.072300 2474 log.cc:824] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:59.075489 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec: No bootstrap required, opened a new log
I20260810 18:53:59.079541 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.057602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36551 (local address 127.29.174.254:35995)
0810 18:53:59.058502 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:59.058516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:59.058879 (+ 363us) server_negotiation.cc:407] Connection header received
0810 18:53:59.060966 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:59.060992 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:59.061482 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:59.061896 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:59.063199 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.065108 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.066283 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.067295 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.067466 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.071368 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:59.071381 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:59.071398 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:59.071451 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:59.075165 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.075982 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.075990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.075998 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.076228 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.076637 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.076642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.076645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.077307 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:59.077645 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:59.077943 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:53:59.078273 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":359,"threads_started":1}
I20260810 18:53:59.083143 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:59.084558 2474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } }
I20260810 18:53:59.084939 2474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:59.085026 2474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce5a359f62b47868f32a891cdb377ec, State: Initialized, Role: FOLLOWER
I20260810 18:53:59.085583 2474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } }
I20260810 18:53:59.085867 2474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:59.086001 2474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:59.086180 2474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:59.088848 2474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } }
I20260810 18:53:59.089330 2474 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce5a359f62b47868f32a891cdb377ec; no voters:
I20260810 18:53:59.089896 2474 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:59.089999 2486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:59.091990 2474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:59.092909 2486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [term 1 LEADER]: Becoming Leader. State: Replica: 7ce5a359f62b47868f32a891cdb377ec, State: Running, Role: LEADER
I20260810 18:53:59.093593 2486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } }
I20260810 18:53:59.101720 2478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.088049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50167 (local address 127.29.174.254:35995)
0810 18:53:59.088220 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:59.088226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:59.088637 (+ 411us) server_negotiation.cc:407] Connection header received
0810 18:53:59.088754 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:59.088760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:59.088828 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:59.088934 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:59.090415 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.091378 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.092691 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.093454 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.093592 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.096173 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:59.096178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:59.096181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:59.096215 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:59.098800 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.099508 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.099511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.099514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.099618 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.100036 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.100039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.100042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.100365 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:59.100610 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:59.101132 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 18:53:59.101468 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:59.104259 2487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce5a359f62b47868f32a891cdb377ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } } }
I20260810 18:53:59.104341 2489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce5a359f62b47868f32a891cdb377ec. Latest consensus state: current_term: 1 leader_uuid: "7ce5a359f62b47868f32a891cdb377ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5a359f62b47868f32a891cdb377ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 35995 } } }
I20260810 18:53:59.104682 2487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:59.104681 2489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:59.106003 2503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:59.110962 2503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:59.118484 2503 catalog_manager.cc:1347] Generated new cluster ID: 88aa55835ff6469eb11cfca3bb5c753b
I20260810 18:53:59.118583 2503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:59.133462 2503 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:59.134595 2503 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:59.140200 2503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce5a359f62b47868f32a891cdb377ec: Generated new TSK 0
I20260810 18:53:59.140872 2503 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:59.152688 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2364
I20260810 18:53:59.153716 2466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:53:59.240173 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2364
W20260810 18:53:59.262284 2343 connection.cc:535] client connection to 127.29.174.254:35995 recv error: Network error: failed to read from TLS socket (remote: 127.29.174.254:35995): Connection reset by peer (error 104)
2026-08-10T18:53:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:59 dist-test-slave-6zbq krb5kdc[2527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:59 dist-test-slave-6zbq krb5kdc[2527](info): set up 2 sockets
Aug 10 18:53:59 dist-test-slave-6zbq krb5kdc[2527](info): commencing operation
krb5kdc: starting...
W20260810 18:54:00.476038 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.005s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:00 dist-test-slave-6zbq krb5kdc[2527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:00.742727 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36805
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:36805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:01.057783 2546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:01.058028 2546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:01.058145 2546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:01.065033 2546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:01.065135 2546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:01.065218 2546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:01.065304 2546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:01.065362 2546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:01.074620 2546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:36805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:36805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:01.075690 2546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:01.076859 2546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:01.087647 2558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:01.087752 2556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:01.087749 2561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:01.088610 2546 server_base.cc:1034] running on GCE node
I20260810 18:54:01.089298 2546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:01.090544 2546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:01.091745 2546 hybrid_clock.cc:648] HybridClock initialized: now 1786388041091703 us; error 35 us; skew 500 ppm
I20260810 18:54:01.092047 2546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:01 dist-test-slave-6zbq krb5kdc[2527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388041, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:01.094969 2546 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:01.096237 2546 webserver.cc:460] Webserver started at http://127.29.174.254:36779/ using document root <none> and password file <none>
I20260810 18:54:01.096845 2546 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:01.096961 2546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:01.097232 2546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:01.100018 2546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "224ab93f72f44a65861135a22b75895c"
format_stamp: "Formatted at 2026-08-10 18:54:01 on dist-test-slave-6zbq"
server_key: "556b76811ee087a5d397d6a32a397aa8"
server_key_iv: "fc21393718628f6cea9dfb54f836c5f9"
server_key_version: "encryptionkey@0"
I20260810 18:54:01.100735 2546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "224ab93f72f44a65861135a22b75895c"
format_stamp: "Formatted at 2026-08-10 18:54:01 on dist-test-slave-6zbq"
server_key: "556b76811ee087a5d397d6a32a397aa8"
server_key_iv: "fc21393718628f6cea9dfb54f836c5f9"
server_key_version: "encryptionkey@0"
I20260810 18:54:01.105405 2546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:54:01.108798 2573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:01.109997 2546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:54:01.110159 2546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "224ab93f72f44a65861135a22b75895c"
format_stamp: "Formatted at 2026-08-10 18:54:01 on dist-test-slave-6zbq"
server_key: "556b76811ee087a5d397d6a32a397aa8"
server_key_iv: "fc21393718628f6cea9dfb54f836c5f9"
server_key_version: "encryptionkey@0"
I20260810 18:54:01.110332 2546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:01.138422 2546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:01.143155 2546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:01.143463 2546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:01.161137 2546 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:36805
I20260810 18:54:01.161175 2664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:36805 every 8 connection(s)
I20260810 18:54:01.162442 2546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:01.164016 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2546
I20260810 18:54:01.164170 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:01.164619 30395 external_mini_cluster.cc:1428] Setting key 7f415cab34caad8ff9bdfc8900135082
I20260810 18:54:01.166540 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:01.179100 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.166457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58599 (local address 127.29.174.254:36805)
0810 18:54:01.167271 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.167287 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.167334 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:54:01.169411 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.169438 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.169935 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.170388 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.170890 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.172873 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.174107 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.175164 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.175348 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.176417 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:01.176432 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:01.176449 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:01.177163 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:01.177253 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:01.177288 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:01.177462 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:01.177690 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 18:54:01.177859 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":323,"threads_started":1}
I20260810 18:54:01.180364 2667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c: Bootstrap starting.
I20260810 18:54:01.182700 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:01.183547 2667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:01.184727 2667 log.cc:824] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:01.187851 2667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c: No bootstrap required, opened a new log
I20260810 18:54:01.194260 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.187733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56891 (local address 127.29.174.254:36805)
0810 18:54:01.187906 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.187913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.188397 (+ 484us) server_negotiation.cc:407] Connection header received
0810 18:54:01.188570 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.188574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.188639 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.188716 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.189322 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.190075 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.191915 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.192345 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.192438 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.193551 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:01.193556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:01.193559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:01.193574 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:01.193618 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:01.193633 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:01.193721 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:01.193916 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 18:54:01.193954 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:54:01.196638 2667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } }
I20260810 18:54:01.196961 2667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:01.197048 2667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224ab93f72f44a65861135a22b75895c, State: Initialized, Role: FOLLOWER
I20260810 18:54:01.197547 2667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } }
I20260810 18:54:01.197773 2667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:01.197896 2667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:01.198047 2667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:01.200198 2667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } }
I20260810 18:54:01.200666 2667 leader_election.cc:302] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 224ab93f72f44a65861135a22b75895c; no voters:
I20260810 18:54:01.201202 2667 leader_election.cc:288] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:01.201319 2682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:01.201973 2682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [term 1 LEADER]: Becoming Leader. State: Replica: 224ab93f72f44a65861135a22b75895c, State: Running, Role: LEADER
I20260810 18:54:01.202534 2682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } }
I20260810 18:54:01.203083 2667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:01.205292 2687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 224ab93f72f44a65861135a22b75895c. Latest consensus state: current_term: 1 leader_uuid: "224ab93f72f44a65861135a22b75895c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } } }
I20260810 18:54:01.205598 2687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:01.206038 2684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "224ab93f72f44a65861135a22b75895c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224ab93f72f44a65861135a22b75895c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 36805 } } }
I20260810 18:54:01.206326 2684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:01.207120 2694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:01.212138 2694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:01.221415 2694 catalog_manager.cc:1347] Generated new cluster ID: 2ae29f03d37949e0a06169e12f8c6840
I20260810 18:54:01.221524 2694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:01.235476 2694 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:01.236591 2694 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:01.249907 2694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 224ab93f72f44a65861135a22b75895c: Generated new TSK 0
I20260810 18:54:01.250595 2694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:01.258406 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2546
W20260810 18:54:01.259876 2656 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58599
W20260810 18:54:01.260556 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2546: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:54:01.260685 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2546
W20260810 18:54:01.285729 2521 connection.cc:535] client connection to 127.29.174.254:36805 recv error: Network error: failed to read from TLS socket (remote: 127.29.174.254:36805): Connection reset by peer (error 104)
2026-08-10T18:54:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:01 dist-test-slave-6zbq krb5kdc[2717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:01 dist-test-slave-6zbq krb5kdc[2717](info): set up 2 sockets
Aug 10 18:54:01 dist-test-slave-6zbq krb5kdc[2717](info): commencing operation
krb5kdc: starting...
W20260810 18:54:02.501457 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:02 dist-test-slave-6zbq krb5kdc[2717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:02.777514 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33729
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:35997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:33729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:03.097312 2740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:03.097559 2740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:03.097692 2740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:03.104645 2740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:03.104746 2740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:03.104823 2740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:03.104902 2740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:03.104959 2740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:03.114245 2740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:35997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:33729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:33729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:03.115388 2740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:03.116576 2740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:03.127198 2753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:03.127282 2749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:03.127308 2750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:03.128235 2740 server_base.cc:1034] running on GCE node
I20260810 18:54:03.128959 2740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:03.130180 2740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:03.131405 2740 hybrid_clock.cc:648] HybridClock initialized: now 1786388043131357 us; error 38 us; skew 500 ppm
I20260810 18:54:03.131742 2740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:03 dist-test-slave-6zbq krb5kdc[2717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388043, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:03.134721 2740 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:03.136040 2740 webserver.cc:460] Webserver started at http://127.29.174.254:39993/ using document root <none> and password file <none>
I20260810 18:54:03.136646 2740 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:03.136760 2740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:03.137043 2740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:03.139853 2740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "1294f8f3235246d58ecb909d1aaeddda"
format_stamp: "Formatted at 2026-08-10 18:54:03 on dist-test-slave-6zbq"
server_key: "c9adacc2172cad9eb26011d680602de2"
server_key_iv: "08847751247835ccf3588278097e8268"
server_key_version: "encryptionkey@0"
I20260810 18:54:03.140578 2740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "1294f8f3235246d58ecb909d1aaeddda"
format_stamp: "Formatted at 2026-08-10 18:54:03 on dist-test-slave-6zbq"
server_key: "c9adacc2172cad9eb26011d680602de2"
server_key_iv: "08847751247835ccf3588278097e8268"
server_key_version: "encryptionkey@0"
I20260810 18:54:03.145253 2740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:03.148412 2770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:03.149542 2740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:54:03.149695 2740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "1294f8f3235246d58ecb909d1aaeddda"
format_stamp: "Formatted at 2026-08-10 18:54:03 on dist-test-slave-6zbq"
server_key: "c9adacc2172cad9eb26011d680602de2"
server_key_iv: "08847751247835ccf3588278097e8268"
server_key_version: "encryptionkey@0"
I20260810 18:54:03.149863 2740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:03.155061 2740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:03.158887 2740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:03.159135 2740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:03.175451 2740 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:33729
I20260810 18:54:03.175485 2881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:33729 every 8 connection(s)
I20260810 18:54:03.176617 2740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:03.179943 2882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:03.180150 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2740
I20260810 18:54:03.180303 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:03.180716 30395 external_mini_cluster.cc:1428] Setting key e38786e83d0687b4984a3bfcaa4a07c8
I20260810 18:54:03.188272 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:03.182249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43391 (local address 127.29.174.254:33729)
0810 18:54:03.183106 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:03.183124 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:03.183175 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:54:03.184731 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:03.184758 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:03.185146 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:03.185550 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:03.186492 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:03.186500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:03.186516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:03.186531 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:03.186619 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:03.186647 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:03.186871 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:03.187003 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:54:03.187173 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":325,"threads_started":1}
I20260810 18:54:03.191459 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:03.193620 2882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda: Bootstrap starting.
I20260810 18:54:03.197199 2882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:03.198330 2882 log.cc:824] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:03.198424 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:03.196363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36341 (local address 127.29.174.254:33729)
0810 18:54:03.196513 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:03.196518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:03.196816 (+ 298us) server_negotiation.cc:407] Connection header received
0810 18:54:03.196921 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:03.196927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:03.196980 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:03.197090 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:03.197819 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:03.197824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:03.197829 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:03.197845 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:03.197886 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:03.197905 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:03.198018 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:03.198076 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 18:54:03.198132 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:54:03.200947 2882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda: No bootstrap required, opened a new log
I20260810 18:54:03.208027 2882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } }
I20260810 18:54:03.208356 2882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:03.208455 2882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1294f8f3235246d58ecb909d1aaeddda, State: Initialized, Role: FOLLOWER
I20260810 18:54:03.208985 2882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } }
I20260810 18:54:03.209231 2882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:03.209384 2882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:03.209553 2882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:03.211622 2882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } }
I20260810 18:54:03.212044 2882 leader_election.cc:302] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1294f8f3235246d58ecb909d1aaeddda; no voters:
I20260810 18:54:03.212574 2882 leader_election.cc:288] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:03.212679 2901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:03.213253 2901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [term 1 LEADER]: Becoming Leader. State: Replica: 1294f8f3235246d58ecb909d1aaeddda, State: Running, Role: LEADER
I20260810 18:54:03.213750 2901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } }
I20260810 18:54:03.214370 2882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:03.216463 2902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1294f8f3235246d58ecb909d1aaeddda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } } }
I20260810 18:54:03.216432 2904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1294f8f3235246d58ecb909d1aaeddda. Latest consensus state: current_term: 1 leader_uuid: "1294f8f3235246d58ecb909d1aaeddda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1294f8f3235246d58ecb909d1aaeddda" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 33729 } } }
I20260810 18:54:03.216904 2904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:03.216908 2902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:03.217881 2913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:03.223119 2913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:03.230852 2913 catalog_manager.cc:1347] Generated new cluster ID: 3ba41f6da7444678929ed61ff03aa5d8
I20260810 18:54:03.230954 2913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:03.243221 2913 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:03.243654 2913 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:03.250262 2913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1294f8f3235246d58ecb909d1aaeddda: Generated new TSK 0
I20260810 18:54:03.251000 2913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:03.256397 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2740
W20260810 18:54:03.257387 2863 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43391
W20260810 18:54:03.258069 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2740: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:54:03.258219 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2740
2026-08-10T18:54:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1996 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:03 dist-test-slave-6zbq krb5kdc[2939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:03 dist-test-slave-6zbq krb5kdc[2939](info): set up 2 sockets
Aug 10 18:54:03 dist-test-slave-6zbq krb5kdc[2939](info): commencing operation
krb5kdc: starting...
W20260810 18:54:04.495522 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:04 dist-test-slave-6zbq krb5kdc[2939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:04.755295 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38899
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:05.065197 2966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:05.065454 2966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:05.065584 2966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:05.072443 2966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:05.072540 2966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:05.072610 2966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:05.072693 2966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:05.072749 2966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:05.081918 2966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:05.082973 2966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:05.084154 2966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:05.094844 2975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:05.094914 2973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:05.095093 2972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:05.095897 2966 server_base.cc:1034] running on GCE node
I20260810 18:54:05.096637 2966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:05.097911 2966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:05.099107 2966 hybrid_clock.cc:648] HybridClock initialized: now 1786388045099067 us; error 28 us; skew 500 ppm
I20260810 18:54:05.099432 2966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:05 dist-test-slave-6zbq krb5kdc[2939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388045, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:05.106578 2966 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:05.107939 2966 webserver.cc:460] Webserver started at http://127.29.174.254:42343/ using document root <none> and password file <none>
I20260810 18:54:05.108541 2966 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:05.108654 2966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:05.108924 2966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:05.111732 2966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "d130ac1a17884306869037c83259d6ec"
format_stamp: "Formatted at 2026-08-10 18:54:05 on dist-test-slave-6zbq"
server_key: "c42a254b74eaaab7768137bf04a71e19"
server_key_iv: "bd06040bf2d2f7fc071d154e72a092c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:05.112442 2966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "d130ac1a17884306869037c83259d6ec"
format_stamp: "Formatted at 2026-08-10 18:54:05 on dist-test-slave-6zbq"
server_key: "c42a254b74eaaab7768137bf04a71e19"
server_key_iv: "bd06040bf2d2f7fc071d154e72a092c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:05.117250 2966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:05.120433 2983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:05.121591 2966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:54:05.121740 2966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "d130ac1a17884306869037c83259d6ec"
format_stamp: "Formatted at 2026-08-10 18:54:05 on dist-test-slave-6zbq"
server_key: "c42a254b74eaaab7768137bf04a71e19"
server_key_iv: "bd06040bf2d2f7fc071d154e72a092c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:05.121907 2966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:05.151800 2966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:05.155663 2966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:05.155908 2966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:05.172748 2966 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:38899
I20260810 18:54:05.172782 3058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:38899 every 8 connection(s)
I20260810 18:54:05.173955 2966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:05.175405 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 2966
I20260810 18:54:05.175563 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:05.175987 30395 external_mini_cluster.cc:1428] Setting key ee000f615ec0809d5cab1d952e8d3433
I20260810 18:54:05.177786 3059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:54:05 dist-test-slave-6zbq krb5kdc[2939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:05.189630 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec: Bootstrap starting.
I20260810 18:54:05.192837 3059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:05.193917 3059 log.cc:824] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:05.196483 3059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec: No bootstrap required, opened a new log
I20260810 18:54:05.197168 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:05.177587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47513 (local address 127.29.174.254:38899)
0810 18:54:05.178377 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:05.178393 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:05.178441 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:54:05.180532 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:05.180558 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:05.181037 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:05.181462 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:05.182957 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.184303 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.185512 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.186128 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.186299 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.190535 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:05.190543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:05.190554 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:05.190590 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:05.193365 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:05.194117 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:05.194124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:05.194129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:05.194293 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:05.194720 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:05.194724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:05.194726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:05.195187 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:05.195453 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:05.195866 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 18:54:05.196221 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":325,"threads_started":1}
I20260810 18:54:05.200484 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:05.203727 3059 raft_consensus.cc:348] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } }
I20260810 18:54:05.204039 3059 raft_consensus.cc:374] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:05.204124 3059 raft_consensus.cc:729] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d130ac1a17884306869037c83259d6ec, State: Initialized, Role: FOLLOWER
I20260810 18:54:05.204607 3059 consensus_queue.cc:260] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } }
I20260810 18:54:05.204921 3059 raft_consensus.cc:388] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:05.205070 3059 raft_consensus.cc:482] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:05.205257 3059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:05.207698 3059 raft_consensus.cc:504] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } }
I20260810 18:54:05.208156 3059 leader_election.cc:302] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d130ac1a17884306869037c83259d6ec; no voters:
I20260810 18:54:05.208678 3059 leader_election.cc:288] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:05.208813 3074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:05.209389 3074 raft_consensus.cc:686] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [term 1 LEADER]: Becoming Leader. State: Replica: d130ac1a17884306869037c83259d6ec, State: Running, Role: LEADER
I20260810 18:54:05.209961 3074 consensus_queue.cc:237] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } }
I20260810 18:54:05.211930 3077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader d130ac1a17884306869037c83259d6ec. Latest consensus state: current_term: 1 leader_uuid: "d130ac1a17884306869037c83259d6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } } }
I20260810 18:54:05.212296 3059 sys_catalog.cc:564] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:05.212343 3077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:05.215408 3079 sys_catalog.cc:455] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d130ac1a17884306869037c83259d6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d130ac1a17884306869037c83259d6ec" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 38899 } } }
I20260810 18:54:05.215840 3086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:05.215689 3079 sys_catalog.cc:458] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:05.220438 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:05.204820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33281 (local address 127.29.174.254:38899)
0810 18:54:05.204965 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:05.204970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:05.205052 (+ 82us) server_negotiation.cc:407] Connection header received
0810 18:54:05.205219 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:05.205225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:05.205297 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:05.205395 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:05.206687 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.207708 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.209148 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.209922 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.210053 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.212770 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:05.212778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:05.212783 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:05.212822 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:05.216442 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:05.217256 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:05.217261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:05.217264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:05.217505 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:05.217997 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:05.218002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:05.218006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:05.218423 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:05.218703 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:05.219808 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0810 18:54:05.220035 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:54:05.220821 3086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:05.230876 3086 catalog_manager.cc:1347] Generated new cluster ID: 17c1206c21b3427f9006560ee3cacd60
I20260810 18:54:05.230984 3086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:05.243266 3086 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:05.244451 3086 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:05.250566 3086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d130ac1a17884306869037c83259d6ec: Generated new TSK 0
I20260810 18:54:05.251247 3086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:05.259910 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 2966
I20260810 18:54:05.260927 3053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:05.347759 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 2966
2026-08-10T18:54:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:05 dist-test-slave-6zbq krb5kdc[3115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:05 dist-test-slave-6zbq krb5kdc[3115](info): set up 2 sockets
Aug 10 18:54:05 dist-test-slave-6zbq krb5kdc[3115](info): commencing operation
krb5kdc: starting...
W20260810 18:54:06.579743 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.005s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:06 dist-test-slave-6zbq krb5kdc[3115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:06.826893 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:44873
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:39913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:07.139622 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:07.139894 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:07.140033 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:07.147008 3134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:07.147121 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:07.147197 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:07.147281 3134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:07.147342 3134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:07.156515 3134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:39913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:07.157605 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:07.158818 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:07.169808 3141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:07.169847 3146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:07.169878 3142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:07.170861 3134 server_base.cc:1034] running on GCE node
I20260810 18:54:07.171568 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:07.172811 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:07.174010 3134 hybrid_clock.cc:648] HybridClock initialized: now 1786388047173969 us; error 28 us; skew 500 ppm
I20260810 18:54:07.174331 3134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:07 dist-test-slave-6zbq krb5kdc[3115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388047, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:07.177203 3134 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:07.178581 3134 webserver.cc:460] Webserver started at http://127.29.174.254:33375/ using document root <none> and password file <none>
I20260810 18:54:07.179176 3134 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:07.179296 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:07.179564 3134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:07.182296 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "d2ac11bab29142188fa288b5a04e582f"
format_stamp: "Formatted at 2026-08-10 18:54:07 on dist-test-slave-6zbq"
server_key: "b8c726ba1676a0f6126f89bbd06edadc"
server_key_iv: "14bf62a9c0c8e2d199d5b150d259cdf6"
server_key_version: "encryptionkey@0"
I20260810 18:54:07.183028 3134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "d2ac11bab29142188fa288b5a04e582f"
format_stamp: "Formatted at 2026-08-10 18:54:07 on dist-test-slave-6zbq"
server_key: "b8c726ba1676a0f6126f89bbd06edadc"
server_key_iv: "14bf62a9c0c8e2d199d5b150d259cdf6"
server_key_version: "encryptionkey@0"
I20260810 18:54:07.187660 3134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:54:07.190739 3154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:07.191847 3134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 18:54:07.192000 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "d2ac11bab29142188fa288b5a04e582f"
format_stamp: "Formatted at 2026-08-10 18:54:07 on dist-test-slave-6zbq"
server_key: "b8c726ba1676a0f6126f89bbd06edadc"
server_key_iv: "14bf62a9c0c8e2d199d5b150d259cdf6"
server_key_version: "encryptionkey@0"
I20260810 18:54:07.192162 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:07.220738 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:07.224476 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:07.224725 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:07.241554 3134 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:44873
I20260810 18:54:07.241595 3252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:44873 every 8 connection(s)
I20260810 18:54:07.242777 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:07.246109 3253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:07.247175 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3134
I20260810 18:54:07.247350 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:07.247778 30395 external_mini_cluster.cc:1428] Setting key 92ed0c903c5c8adc3845a391fa44f0f6
I20260810 18:54:07.257566 3253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: Bootstrap starting.
I20260810 18:54:07.259711 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:07.249403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39157 (local address 127.29.174.254:44873)
0810 18:54:07.250171 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:07.250182 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:07.250227 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:54:07.251987 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:07.252008 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:07.252361 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:07.252700 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:07.253252 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.254526 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.255783 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.256413 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.256586 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.257610 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:07.257620 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:07.257636 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:07.258155 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:07.258228 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:07.258253 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:07.258378 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:07.258586 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 18:54:07.258829 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":339,"threads_started":1}
I20260810 18:54:07.260820 3253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:07.261870 3253 log.cc:824] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:07.263271 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:07.264448 3253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: No bootstrap required, opened a new log
I20260810 18:54:07.271867 3253 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } }
I20260810 18:54:07.272220 3253 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:07.272324 3253 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2ac11bab29142188fa288b5a04e582f, State: Initialized, Role: FOLLOWER
I20260810 18:54:07.272765 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:07.267266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60273 (local address 127.29.174.254:44873)
0810 18:54:07.267395 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:07.267399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:07.267861 (+ 462us) server_negotiation.cc:407] Connection header received
0810 18:54:07.268029 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:07.268032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:07.268086 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:07.268161 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:07.268640 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.269343 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.270638 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.271078 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.271187 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.272128 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:07.272133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:07.272136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:07.272151 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:07.272187 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:07.272205 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:07.272296 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:07.272415 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 18:54:07.272495 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:54:07.272925 3253 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } }
I20260810 18:54:07.273243 3253 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:07.273397 3253 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:07.273559 3253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:07.275700 3253 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } }
I20260810 18:54:07.276131 3253 leader_election.cc:302] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2ac11bab29142188fa288b5a04e582f; no voters:
I20260810 18:54:07.276618 3253 leader_election.cc:288] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:07.276736 3268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:07.277397 3268 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [term 1 LEADER]: Becoming Leader. State: Replica: d2ac11bab29142188fa288b5a04e582f, State: Running, Role: LEADER
I20260810 18:54:07.277928 3268 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } }
I20260810 18:54:07.278472 3253 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:07.285190 3270 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2ac11bab29142188fa288b5a04e582f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } } }
I20260810 18:54:07.285567 3270 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:07.286005 3272 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2ac11bab29142188fa288b5a04e582f. Latest consensus state: current_term: 1 leader_uuid: "d2ac11bab29142188fa288b5a04e582f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ac11bab29142188fa288b5a04e582f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 44873 } } }
I20260810 18:54:07.286398 3272 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f [sys.catalog]: This master's current role is: LEADER
W20260810 18:54:07.289296 3292 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:54:07.289409 3292 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:54:07.289539 3299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:07.291198 3299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:07.298463 3299 catalog_manager.cc:1347] Generated new cluster ID: ec765bbae42d4f3da6e38703bc232619
I20260810 18:54:07.298556 3299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:07.312242 3299 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:07.313413 3299 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:07.319053 3299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2ac11bab29142188fa288b5a04e582f: Generated new TSK 0
I20260810 18:54:07.319726 3299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:07.325677 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3134
W20260810 18:54:07.326851 3245 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39157
W20260810 18:54:07.327540 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3134: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:54:07.327696 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3134
2026-08-10T18:54:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:07 dist-test-slave-6zbq krb5kdc[3310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:07 dist-test-slave-6zbq krb5kdc[3310](info): set up 2 sockets
Aug 10 18:54:07 dist-test-slave-6zbq krb5kdc[3310](info): commencing operation
krb5kdc: starting...
W20260810 18:54:08.564445 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:08 dist-test-slave-6zbq krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:08.845372 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:42897
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:45653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:09.172627 3334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:09.172884 3334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:09.173018 3334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:09.179883 3334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:09.179973 3334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:09.180047 3334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:09.180130 3334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:09.180188 3334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:09.189435 3334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:45653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:42897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:42897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:09.190552 3334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:09.191707 3334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:09.202559 3351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:09.202667 3347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:09.202641 3349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:09.203691 3334 server_base.cc:1034] running on GCE node
I20260810 18:54:09.204416 3334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:09.205684 3334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:09.206887 3334 hybrid_clock.cc:648] HybridClock initialized: now 1786388049206846 us; error 27 us; skew 500 ppm
I20260810 18:54:09.207211 3334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:09 dist-test-slave-6zbq krb5kdc[3310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388049, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:09.210357 3334 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:09.211704 3334 webserver.cc:460] Webserver started at http://127.29.174.254:33805/ using document root <none> and password file <none>
I20260810 18:54:09.212292 3334 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:09.212389 3334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:09.212625 3334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:09.215438 3334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "c8f90c781a294b1aa37bbcbb1fc8676f"
format_stamp: "Formatted at 2026-08-10 18:54:09 on dist-test-slave-6zbq"
server_key: "dae4fd846ed515fd362c322c37a00c35"
server_key_iv: "76057708a89307df0f87bb1363da67ba"
server_key_version: "encryptionkey@0"
I20260810 18:54:09.216125 3334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "c8f90c781a294b1aa37bbcbb1fc8676f"
format_stamp: "Formatted at 2026-08-10 18:54:09 on dist-test-slave-6zbq"
server_key: "dae4fd846ed515fd362c322c37a00c35"
server_key_iv: "76057708a89307df0f87bb1363da67ba"
server_key_version: "encryptionkey@0"
I20260810 18:54:09.220772 3334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:54:09.223874 3361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:09.224993 3334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:54:09.225145 3334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "c8f90c781a294b1aa37bbcbb1fc8676f"
format_stamp: "Formatted at 2026-08-10 18:54:09 on dist-test-slave-6zbq"
server_key: "dae4fd846ed515fd362c322c37a00c35"
server_key_iv: "76057708a89307df0f87bb1363da67ba"
server_key_version: "encryptionkey@0"
I20260810 18:54:09.225322 3334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:09.230504 3334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:09.233850 3334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:09.234099 3334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:09.250746 3334 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:42897
I20260810 18:54:09.250777 3445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:42897 every 8 connection(s)
I20260810 18:54:09.251971 3334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:09.253444 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3334
I20260810 18:54:09.253600 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:09.253999 30395 external_mini_cluster.cc:1428] Setting key f0ced7ae44ff3fd71c0618061d8a261f
I20260810 18:54:09.255941 3448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:09.261122 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:09.255565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46819 (local address 127.29.174.254:42897)
0810 18:54:09.256307 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:09.256318 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:09.256358 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:54:09.258022 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:09.258042 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:09.258389 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:09.258704 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:09.259707 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:09.259713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:09.259728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:09.259742 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:09.259815 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:09.259838 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:09.259993 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:09.260086 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 18:54:09.260238 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":339,"threads_started":1}
I20260810 18:54:09.264398 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:09.268025 3448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f: Bootstrap starting.
I20260810 18:54:09.271381 3448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:09.271535 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:09.269277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41461 (local address 127.29.174.254:42897)
0810 18:54:09.269443 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:09.269449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:09.269906 (+ 457us) server_negotiation.cc:407] Connection header received
0810 18:54:09.270012 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:09.270017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:09.270068 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:09.270165 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:09.270947 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:09.270952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:09.270956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:09.270974 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:09.271016 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:09.271035 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:09.271149 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:09.271194 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 18:54:09.271243 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:54:09.272557 3448 log.cc:824] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:09.275295 3448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f: No bootstrap required, opened a new log
I20260810 18:54:09.282179 3448 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } }
I20260810 18:54:09.282555 3448 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:09.282650 3448 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f90c781a294b1aa37bbcbb1fc8676f, State: Initialized, Role: FOLLOWER
I20260810 18:54:09.283174 3448 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } }
I20260810 18:54:09.283376 3448 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:09.283509 3448 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:09.283661 3448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:09.285699 3448 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } }
I20260810 18:54:09.286144 3448 leader_election.cc:302] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8f90c781a294b1aa37bbcbb1fc8676f; no voters:
I20260810 18:54:09.286717 3448 leader_election.cc:288] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:09.286821 3467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:09.287985 3467 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [term 1 LEADER]: Becoming Leader. State: Replica: c8f90c781a294b1aa37bbcbb1fc8676f, State: Running, Role: LEADER
I20260810 18:54:09.288427 3467 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } }
I20260810 18:54:09.288717 3448 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:09.291018 3469 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } } }
I20260810 18:54:09.291399 3469 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:09.294593 3472 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8f90c781a294b1aa37bbcbb1fc8676f. Latest consensus state: current_term: 1 leader_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f90c781a294b1aa37bbcbb1fc8676f" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 42897 } } }
I20260810 18:54:09.294992 3472 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:09.295313 3482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:09.299644 3482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:09.307444 3482 catalog_manager.cc:1347] Generated new cluster ID: e9c894215d3e4d2e93f22113fe050164
I20260810 18:54:09.307556 3482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:09.325701 3482 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:09.326052 3482 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:09.333730 3482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8f90c781a294b1aa37bbcbb1fc8676f: Generated new TSK 0
I20260810 18:54:09.334429 3482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:09.345678 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3334
W20260810 18:54:09.346865 3438 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46819
W20260810 18:54:09.347520 30395 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3334: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:54:09.347651 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3334
2026-08-10T18:54:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2017 ms)
[----------] 18 tests from AuthTokenIssuingTest (36845 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:09 dist-test-slave-6zbq krb5kdc[3500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:09 dist-test-slave-6zbq krb5kdc[3500](info): set up 2 sockets
Aug 10 18:54:09 dist-test-slave-6zbq krb5kdc[3500](info): commencing operation
krb5kdc: starting...
W20260810 18:54:10.584681 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:10 dist-test-slave-6zbq krb5kdc[3500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:10.855343 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45503
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:11.176381 3528 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:11.176682 3528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:11.176781 3528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:11.176899 3528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:11.183862 3528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:11.183941 3528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:11.184003 3528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:11.184074 3528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:11.184155 3528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:11.193814 3528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:45503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:11.195055 3528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:11.196307 3528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:11.207654 3536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:11.207620 3539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:11.207680 3542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:11.208585 3528 server_base.cc:1034] running on GCE node
I20260810 18:54:11.209302 3528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:11.210559 3528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:11.211767 3528 hybrid_clock.cc:648] HybridClock initialized: now 1786388051211701 us; error 44 us; skew 500 ppm
I20260810 18:54:11.212069 3528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:11.218758 3528 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:11.220122 3528 webserver.cc:460] Webserver started at http://127.29.174.254:38139/ using document root <none> and password file <none>
I20260810 18:54:11.220781 3528 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:11.220871 3528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:11.221261 3528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:11.224175 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "dece5d9d74daa7171ad1aac52d723fb0"
server_key_iv: "96e6bc320b9c66df9a8359f8d694d360"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.224943 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "dece5d9d74daa7171ad1aac52d723fb0"
server_key_iv: "96e6bc320b9c66df9a8359f8d694d360"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.229712 3528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:11.232877 3563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:11.234040 3528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:54:11.234203 3528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "dece5d9d74daa7171ad1aac52d723fb0"
server_key_iv: "96e6bc320b9c66df9a8359f8d694d360"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.234436 3528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:11.255306 3528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:11.258833 3528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:11.259057 3528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:11.276175 3528 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:45503
I20260810 18:54:11.276192 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:45503 every 8 connection(s)
I20260810 18:54:11.277384 3528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:11.280769 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:11.286758 3660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:11.287560 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3528
I20260810 18:54:11.287786 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:11.288587 30395 external_mini_cluster.cc:1428] Setting key f4e477b75ef08d3d30fb80ef0758159a
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:11.308051 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.290055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45503 (local address 127.0.0.1:38434)
0810 18:54:11.290870 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.290914 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.292778 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.293524 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.293531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.294178 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.294535 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.294554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.296030 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.296034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.297050 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.297063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.297939 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.297947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.298111 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.298708 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.298734 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.300887 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:11.304189 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.304201 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.304213 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.304577 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.304958 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.304963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.304967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.305116 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.305810 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:11.305822 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:11.306027 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 18:54:11.306240 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 18:54:11.306861 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":138,"thread_start_us":328,"threads_started":1}
I20260810 18:54:11.308072 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.290057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.29.174.254:45503)
0810 18:54:11.291595 (+ 1538us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.291603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.291655 (+ 52us) server_negotiation.cc:407] Connection header received
0810 18:54:11.293110 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.293115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.293320 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.293494 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.294723 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.295889 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.297186 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.297711 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.297881 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.301105 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:11.301113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:11.301124 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:11.301156 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:11.304049 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.304704 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.304712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.304715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.304856 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.305227 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.305230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.305233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.305683 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:11.305876 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:11.306435 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 18:54:11.306693 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1330,"thread_start_us":1138,"threads_started":1}
I20260810 18:54:11.310961 3660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:11.311913 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.311562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45682)
0810 18:54:11.311713 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.311811 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:54:11.312122 3566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
I20260810 18:54:11.313505 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.313191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45692)
0810 18:54:11.313329 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.313390 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:54:11.313678 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:36217: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.174.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
I20260810 18:54:11.351464 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:36217
--webserver_interface=127.29.174.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:54:11.362972 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } attempt: 1
I20260810 18:54:11.363988 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.363563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45704)
0810 18:54:11.363751 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.363855 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:54:11.364988 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.364666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45706)
0810 18:54:11.364799 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.364865 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:54:11.365180 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:36217: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
I20260810 18:54:11.433387 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } attempt: 2
I20260810 18:54:11.434556 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.434087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45708)
0810 18:54:11.434303 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.434419 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 18:54:11.435608 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.435201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45722)
0810 18:54:11.435379 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.435465 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 18:54:11.435820 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:36217: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
I20260810 18:54:11.527042 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } attempt: 3
I20260810 18:54:11.528204 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.527791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45736)
0810 18:54:11.527981 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.528074 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:54:11.529139 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.528827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45746)
0810 18:54:11.528939 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.529006 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 18:54:11.529326 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:36217: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
W20260810 18:54:11.670763 3670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:11.671044 3670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:11.671139 3670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:11.671248 3670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:54:11.672544 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } attempt: 4
I20260810 18:54:11.673733 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.673215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45758)
0810 18:54:11.673507 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.673599 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260810 18:54:11.674657 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.674330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45770)
0810 18:54:11.674462 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.674543 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 18:54:11.674831 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:36217: Network error: Client connection negotiation failed: client connection to 127.29.174.253:36217: connect: Connection refused (error 111)
W20260810 18:54:11.678728 3670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:11.678809 3670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:11.678895 3670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:11.678957 3670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:11.679037 3670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:11.688705 3670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:36217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.174.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:11.689810 3670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:11.691001 3670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:11.701933 3686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:11.701983 3682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:11.702397 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:11.702842 3670 server_base.cc:1034] running on GCE node
I20260810 18:54:11.703517 3670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:11.704761 3670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:11.705967 3670 hybrid_clock.cc:648] HybridClock initialized: now 1786388051705904 us; error 35 us; skew 500 ppm
I20260810 18:54:11.706264 3670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:11.709098 3670 init.cc:377] Logged in from keytab as kudu/127.29.174.253@KRBTEST.COM (short username kudu)
I20260810 18:54:11.710398 3670 webserver.cc:460] Webserver started at http://127.29.174.253:46251/ using document root <none> and password file <none>
I20260810 18:54:11.711015 3670 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:11.711104 3670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:11.711403 3670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:11.714118 3670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data/instance:
uuid: "4e74dbaad8784c08b3db8181c907f6bf"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "d4a62b34ede8b5fedd92431b3e7742ba"
server_key_iv: "283f220a08121f9c615ef64c5fff0362"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.714888 3670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal/instance:
uuid: "4e74dbaad8784c08b3db8181c907f6bf"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "d4a62b34ede8b5fedd92431b3e7742ba"
server_key_iv: "283f220a08121f9c615ef64c5fff0362"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.719617 3670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 18:54:11.722796 3700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:11.723971 3670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 18:54:11.724123 3670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "4e74dbaad8784c08b3db8181c907f6bf"
format_stamp: "Formatted at 2026-08-10 18:54:11 on dist-test-slave-6zbq"
server_key: "d4a62b34ede8b5fedd92431b3e7742ba"
server_key_iv: "283f220a08121f9c615ef64c5fff0362"
server_key_version: "encryptionkey@0"
I20260810 18:54:11.724323 3670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:11.753556 3670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:11.757148 3670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:11.757386 3670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:11.774284 3670 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.253:36217
I20260810 18:54:11.774291 3799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.253:36217 every 8 connection(s)
I20260810 18:54:11.775492 3670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
I20260810 18:54:11.777493 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3670
I20260810 18:54:11.777645 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-1/wal/instance
I20260810 18:54:11.778067 30395 external_mini_cluster.cc:1428] Setting key fe8c011ec7c29fd4f7b86931145d6890
I20260810 18:54:11.778980 3800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:11.785043 3800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:11.807343 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.787431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.29.174.254:45503)
0810 18:54:11.787635 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.787641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.788977 (+ 1336us) server_negotiation.cc:407] Connection header received
0810 18:54:11.790919 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.790926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.791008 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.791095 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.793018 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.793698 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.795847 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.796289 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.796393 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.800609 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:11.800615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:11.800618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:11.800654 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:11.803305 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.804238 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.804242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.804244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.804368 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.804849 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.804853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.804855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.805169 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:11.805293 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:11.806535 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0810 18:54:11.806850 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:54:11.808166 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.787965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45503 (local address 127.0.0.1:38446)
0810 18:54:11.788813 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.788859 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.790645 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.791359 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.791371 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.792079 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.792827 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.792851 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.793871 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.793877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.795639 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.795661 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.796522 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.796529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.796822 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.797576 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.797613 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.800335 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:11.803452 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.803461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.803477 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.804130 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.804464 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.804468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.804471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.804690 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.805311 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:11.805321 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:11.805819 (+ 498us) client_negotiation.cc:769] Sending connection context
0810 18:54:11.806128 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 18:54:11.806691 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":395,"threads_started":1}
I20260810 18:54:11.810025 3800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.174.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.252@KRBTEST.COM" created.
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:11.825241 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.810897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45774)
0810 18:54:11.811093 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.811112 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.811219 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.812004 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.812007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.812380 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.812522 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.812530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.813913 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.813916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.814851 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.814864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.815882 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.815886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.816012 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.816585 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.816601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.818588 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:11.822424 (+ 3836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.822428 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.822431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.822824 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.823348 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.823354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.823357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.823640 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.824485 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:11.824489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:11.824616 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 18:54:11.824735 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:54:11.824952 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:54:11.825554 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.811010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.29.174.253:36217)
0810 18:54:11.811590 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.811596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.811622 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:54:11.811670 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.811676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.811858 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.811975 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.812664 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.813789 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.814988 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.815750 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.815872 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.818821 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:11.818836 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:11.818852 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:11.818886 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:11.822209 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.822956 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.822965 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.822968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.823107 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.823836 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.823841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.823844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.824331 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:11.824534 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:11.824913 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 18:54:11.825157 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":433,"thread_start_us":277,"threads_started":1}
I20260810 18:54:11.827857 3800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:11.828900 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.828447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53156)
0810 18:54:11.828643 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.828756 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 18:54:11.829126 3706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:11.830683 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.830283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53158)
0810 18:54:11.830461 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.830558 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:54:11.830962 3800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
I20260810 18:54:11.852684 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:44503
--webserver_interface=127.29.174.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:54:11.880272 3800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 1
I20260810 18:54:11.881443 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.880934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53160)
0810 18:54:11.881141 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.881268 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:54:11.882419 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.882026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53168)
0810 18:54:11.882166 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.882262 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 18:54:11.882656 3800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:11.950891 3800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 2
I20260810 18:54:11.952157 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.951586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53170)
0810 18:54:11.951824 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.951951 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 18:54:11.953166 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.952789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53178)
0810 18:54:11.952931 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.953022 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:54:11.953431 3800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:11.974083 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } attempt: 5
Aug 10 18:54:11 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:11.987612 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.974988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.29.174.253:36217)
0810 18:54:11.975156 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.975161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.975181 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:54:11.975244 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.975248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.975307 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.975384 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.976371 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.977050 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.978691 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.979106 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.979192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.982689 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:11.982694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:11.982697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:11.982726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:11.985085 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.985873 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.985877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.985879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.985997 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:11.986388 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:11.986393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:11.986395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:11.986655 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:11.986766 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:11.987213 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 18:54:11.987399 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:54:11.987776 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.974774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45780)
0810 18:54:11.974977 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.974998 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.975126 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.975443 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.975448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.975966 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.976221 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.976236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.977184 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.977191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.978560 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.978577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.979252 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.979258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.979487 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.980071 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.980094 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.982517 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:11.985240 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.985245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.985248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.985775 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.986095 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:11.986100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:11.986104 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:11.986268 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:11.986773 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:11.986778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:11.986960 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 18:54:11.987113 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 18:54:11.987398 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:54:11.988865 3660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:11.989871 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.989447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53194)
0810 18:54:11.989609 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.989698 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 18:54:11.990782 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.990429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53208)
0810 18:54:11.990557 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.990657 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 18:54:11.990998 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:12.042191 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 1
I20260810 18:54:12.043318 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.042870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53222)
0810 18:54:12.043060 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.043147 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:54:12.044245 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.043954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53228)
0810 18:54:12.044069 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.044133 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:54:12.044454 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:12.044641 3800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 3
I20260810 18:54:12.045570 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.045083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53232)
0810 18:54:12.045276 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.045380 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:54:12.046705 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.046192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53234)
0810 18:54:12.046451 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.046551 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260810 18:54:12.046931 3800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:12.112635 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 2
I20260810 18:54:12.113766 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.113318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53238)
0810 18:54:12.113519 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.113602 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:54:12.114705 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.114411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53254)
0810 18:54:12.114520 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.114597 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 18:54:12.114902 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
W20260810 18:54:12.177326 3808 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:12.177603 3808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:12.177672 3808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:12.177752 3808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:12.184706 3808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:12.184782 3808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:12.184835 3808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:12.184885 3808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:12.184943 3808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:12.190115 3800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 4
I20260810 18:54:12.191241 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.190704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53260)
0810 18:54:12.190951 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.191055 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 18:54:12.192433 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.192028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53270)
0810 18:54:12.192234 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.192312 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260810 18:54:12.192694 3800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:12.194905 3808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:45503,127.29.174.253:36217,127.29.174.252:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:44503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.174.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:12.196005 3808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:12.197171 3808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:12.207937 3815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:12.207912 3820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:12.208029 3824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:12.208557 3808 server_base.cc:1034] running on GCE node
I20260810 18:54:12.209317 3808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:12.210577 3808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:12.211782 3808 hybrid_clock.cc:648] HybridClock initialized: now 1786388052211714 us; error 45 us; skew 500 ppm
I20260810 18:54:12.212066 3808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:12.214944 3808 init.cc:377] Logged in from keytab as kudu/127.29.174.252@KRBTEST.COM (short username kudu)
I20260810 18:54:12.216181 3808 webserver.cc:460] Webserver started at http://127.29.174.252:42885/ using document root <none> and password file <none>
I20260810 18:54:12.216869 3808 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:12.216974 3808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:12.217249 3808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:12.219971 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data/instance:
uuid: "a5e9c427117346cdaf517927bbcdd139"
format_stamp: "Formatted at 2026-08-10 18:54:12 on dist-test-slave-6zbq"
server_key: "94f475a9eb1aac37f5a42dd59d63d264"
server_key_iv: "899cc57311fa0f19fb63fbba35c3a724"
server_key_version: "encryptionkey@0"
I20260810 18:54:12.220733 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal/instance:
uuid: "a5e9c427117346cdaf517927bbcdd139"
format_stamp: "Formatted at 2026-08-10 18:54:12 on dist-test-slave-6zbq"
server_key: "94f475a9eb1aac37f5a42dd59d63d264"
server_key_iv: "899cc57311fa0f19fb63fbba35c3a724"
server_key_version: "encryptionkey@0"
I20260810 18:54:12.221091 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 3
I20260810 18:54:12.222004 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.221577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53286)
0810 18:54:12.221777 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.221855 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 18:54:12.222975 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.222604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53290)
0810 18:54:12.222803 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.222870 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 18:54:12.223217 3660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:44503: Network error: Client connection negotiation failed: client connection to 127.29.174.252:44503: connect: Connection refused (error 111)
I20260810 18:54:12.225571 3808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:12.228729 3840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:12.229893 3808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:54:12.230036 3808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "a5e9c427117346cdaf517927bbcdd139"
format_stamp: "Formatted at 2026-08-10 18:54:12 on dist-test-slave-6zbq"
server_key: "94f475a9eb1aac37f5a42dd59d63d264"
server_key_iv: "899cc57311fa0f19fb63fbba35c3a724"
server_key_version: "encryptionkey@0"
I20260810 18:54:12.230214 3808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:12.244124 3808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:12.247992 3808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:12.248251 3808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:12.266214 3808 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.252:44503
I20260810 18:54:12.266278 3932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.252:44503 every 8 connection(s)
I20260810 18:54:12.267627 3808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
I20260810 18:54:12.269042 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 3808
I20260810 18:54:12.269191 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387695213813-30395-0/minicluster-data/master-2/wal/instance
I20260810 18:54:12.269619 30395 external_mini_cluster.cc:1428] Setting key bede5f83c130861ddf8e07ffb749f84e
I20260810 18:54:12.271670 3935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:12.279497 3935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:12.293604 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.271439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53298 (local address 127.29.174.252:44503)
0810 18:54:12.272234 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.272245 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.272285 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:54:12.273830 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.273849 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.274213 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.274560 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.276346 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.278389 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.280086 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.281101 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.281283 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.285909 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.285919 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.285932 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.285979 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.289652 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.290369 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.290378 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.290387 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.290560 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.290918 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.290923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.290926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.291529 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.291734 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.292255 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.292602 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":356,"threads_started":1}
I20260810 18:54:12.296759 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.282058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:45503 (local address 127.0.0.1:38462)
0810 18:54:12.282823 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.282876 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.283102 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.283485 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.283491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.284241 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.284555 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.284566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.285562 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.285567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.287113 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.287129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.287969 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.287976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.288161 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.288737 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.288768 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.291226 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:12.294217 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.294234 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.294240 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.294761 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.295103 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.295108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.295112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.295243 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.295738 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:12.295754 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:12.295944 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 18:54:12.296122 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 18:54:12.296388 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":374,"threads_started":1}
I20260810 18:54:12.297290 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.282051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.29.174.254:45503)
0810 18:54:12.282239 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.282245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.282967 (+ 722us) server_negotiation.cc:407] Connection header received
0810 18:54:12.283241 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.283246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.283320 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.283413 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.284692 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.285423 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.287259 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.287781 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.287888 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.291400 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.291405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.291408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.291438 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.294095 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.294867 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.294871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.294873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.294995 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.295354 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.295357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.295360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.295598 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.295744 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.296258 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.296485 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:54:12.298544 3935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:12.312254 3943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.296648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.29.174.254:45503)
0810 18:54:12.297299 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.297304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.297327 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:54:12.297373 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.297378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.297442 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.297538 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.298663 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.299685 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.301164 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.301648 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.301734 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.306842 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.306847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.306850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.306882 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.309383 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.310134 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.310140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.310143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.310277 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.310665 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.310669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.310671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.310997 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.311241 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.311634 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.311856 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":291,"threads_started":1}
I20260810 18:54:12.312618 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.299239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45782 (local address 127.29.174.253:36217)
0810 18:54:12.299400 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.299404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.299584 (+ 180us) server_negotiation.cc:407] Connection header received
0810 18:54:12.299680 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.299685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.299776 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.299860 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.300599 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.301317 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.303018 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.303467 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.303570 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.307091 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.307096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.307099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.307129 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.309840 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.310624 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.310629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.310632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.310767 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.311133 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.311138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.311141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.311478 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.311612 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.312079 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.312285 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:54:12.313309 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.299267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:36217 (local address 127.0.0.1:45782)
0810 18:54:12.299498 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.299511 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.299590 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.299894 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.299898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.300313 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.300483 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.300493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.301448 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.301452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.302623 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.302638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.303631 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.303638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.304107 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.304760 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.304784 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.306923 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:12.309982 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.309986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.309988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.310519 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.310871 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.310876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.310880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.311037 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.311607 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:12.311611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:12.311791 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 18:54:12.312825 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0810 18:54:12.313032 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260810 18:54:12.314455 3935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388052, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:12.325877 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.315010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53300)
0810 18:54:12.315151 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.315168 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.315268 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.315510 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.315515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.315845 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.315978 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.315989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.316886 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.316890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.317697 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.317707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.318397 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.318404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.318589 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.319152 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.319167 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.321020 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:12.323792 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.323796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.323801 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.324125 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.324445 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.324450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.324454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.324570 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.325013 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:12.325018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:12.325144 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 18:54:12.325270 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 18:54:12.325525 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:54:12.326040 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.315093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.29.174.252:44503)
0810 18:54:12.315224 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.315229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.315245 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:54:12.315359 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.315363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.315414 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.315482 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.316096 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.316758 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.317828 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.318220 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.318314 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.321172 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.321177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.321180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.321205 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.323674 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.324223 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.324228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.324230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.324341 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.324662 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.324668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.324670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.324908 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.325026 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.325356 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.325689 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":53}
I20260810 18:54:12.327144 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.314058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.29.174.253:36217)
0810 18:54:12.314185 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.314188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.314202 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:12.314237 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.314248 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.314296 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.314398 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.315991 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.316698 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.317847 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.318239 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.318384 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.322539 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.322543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.322547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.322584 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.324913 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.325491 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.325495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.325498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.325620 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.325962 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.325967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.325970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.326221 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.326518 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.326766 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.326918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:54:12.331141 3935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: Bootstrap starting.
I20260810 18:54:12.331471 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:12.334448 3935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:12.335738 3935 log.cc:824] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:12.334022 3608 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38474
I20260810 18:54:12.338739 3935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: No bootstrap required, opened a new log
I20260810 18:54:12.349069 3935 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.349445 3935 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e9c427117346cdaf517927bbcdd139, State: Initialized, Role: FOLLOWER
I20260810 18:54:12.349905 3935 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.350759 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.351055 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.351768 3935 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:54:12.360663 3966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:54:12.360782 3966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:54:12.358793 3808 master_runner.cc:186] Error getting master registration for 127.29.174.254:45503: OK, instance_id { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" instance_seqno: 1786388051271424 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:54:12.366387 3808 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:54:12.400408 3660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 4
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:54:12.414892 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.401189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53312)
0810 18:54:12.401410 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.401432 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.401560 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.401927 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.401932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.402510 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.402725 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.402740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.403761 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.403767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.405187 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.405204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.406208 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.406214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.406473 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.407060 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.407084 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.409627 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:12.412577 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.412583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.412587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.412933 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.413270 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.413275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.413279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.413409 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.413935 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:12.413940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:12.414102 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 18:54:12.414222 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:54:12.414508 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 18:54:12.414897 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.401332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.29.174.252:44503)
0810 18:54:12.401517 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.401524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.401548 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:54:12.401692 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.401698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.401774 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.401877 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.402880 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.403636 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.405347 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.406056 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.406174 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.409804 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.409809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.409812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.409846 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.412455 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.413043 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.413047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.413050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.413167 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.413509 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.413514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.413517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.413822 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.413971 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.414335 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.414548 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:54:12.420831 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: Bootstrap starting.
I20260810 18:54:12.424788 3660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:12.426007 3660 log.cc:824] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:12.429296 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: No bootstrap required, opened a new log
I20260810 18:54:12.436715 3660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.437093 3660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5addaeb8b6e74c12aefe5b660b5d5d9d, State: Initialized, Role: FOLLOWER
I20260810 18:54:12.437619 3660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.438535 3972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.438918 3972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.439749 3660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:12.439911 3608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:54:12.440083 3608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:12.442405 3608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.444219 3894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a5e9c427117346cdaf517927bbcdd139"
I20260810 18:54:12.444684 3894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:12.447940 3894 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5addaeb8b6e74c12aefe5b660b5d5d9d in term 1.
I20260810 18:54:12.448283 3608 leader_election.cc:288] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [CANDIDATE]: Term 1 election: Requested vote from peers 4e74dbaad8784c08b3db8181c907f6bf (127.29.174.253:36217), a5e9c427117346cdaf517927bbcdd139 (127.29.174.252:44503)
I20260810 18:54:12.448580 3571 leader_election.cc:302] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5addaeb8b6e74c12aefe5b660b5d5d9d, a5e9c427117346cdaf517927bbcdd139; no voters:
I20260810 18:54:12.449007 3972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:12.461020 3972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [term 1 LEADER]: Becoming Leader. State: Replica: 5addaeb8b6e74c12aefe5b660b5d5d9d, State: Running, Role: LEADER
I20260810 18:54:12.462080 3972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.475831 3983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5addaeb8b6e74c12aefe5b660b5d5d9d. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.476775 3983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:12.489404 3994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:12.491881 3800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } attempt: 5
W20260810 18:54:12.500432 3854 tablet.cc:2369] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 18:54:12 dist-test-slave-6zbq krb5kdc[3500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388051, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:54:12.505342 3994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:12.510233 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.476410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.29.174.253:36217)
0810 18:54:12.476650 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.476657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.487004 (+ 10347us) server_negotiation.cc:407] Connection header received
0810 18:54:12.487500 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.487507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.487605 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.487684 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.489869 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.490957 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.493898 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.494641 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.494740 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.502026 (+ 7286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.502033 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.502038 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.502074 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.505450 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.507074 (+ 1624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.507081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.507084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.507242 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.508100 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.508106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.508110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.508435 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.508673 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.509665 (+ 992us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.509886 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260810 18:54:12.512171 3574 tablet.cc:2369] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:54:12.515028 3943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.483288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.29.174.254:45503)
0810 18:54:12.483476 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.483482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.498918 (+ 15436us) server_negotiation.cc:407] Connection header received
0810 18:54:12.499172 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.499177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.499250 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.499321 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.501140 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.502230 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.504032 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.504695 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.504796 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.507817 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.507823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.507828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.507868 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.511484 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.512261 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.512268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.512272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.512437 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.512886 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.512892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.512896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.513216 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.513442 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.514451 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.514669 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:54:12.517606 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.492384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:44503 (local address 127.0.0.1:53324)
0810 18:54:12.492717 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.492735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.492829 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.493637 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.493643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.494145 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.494394 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.494406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.495823 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.495828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.497250 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.497265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.498423 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.498430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.498654 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.499511 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.499533 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.503552 (+ 4019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:12.514902 (+ 11350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.514908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.514912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.515374 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.515804 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:12.515809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:12.515813 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:12.515999 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:12.516678 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:12.516683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:12.516828 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 18:54:12.516984 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:54:12.517249 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260810 18:54:12.518599 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.493033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.29.174.252:44503)
0810 18:54:12.493199 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.493204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.493221 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:54:12.493288 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.493293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.493363 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.493464 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.494575 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.495663 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.497413 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.498091 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.498202 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.503768 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.506867 (+ 3099us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.506876 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.506953 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.514723 (+ 7770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.515513 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.515518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.515522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.515673 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.516124 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.516129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.516133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.516471 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.516615 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.517939 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.518146 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:54:12.530747 3800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: Bootstrap starting.
I20260810 18:54:12.534440 3800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:12.535629 3800 log.cc:824] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:12.536384 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.499217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.29.174.252:44503)
0810 18:54:12.518804 (+ 19587us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.518809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.518826 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:54:12.518873 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.518878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.518957 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.519048 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.520550 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.521586 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.523387 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.524055 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.524173 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.528863 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:12.528869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:12.528874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:12.528912 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:12.532502 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.533295 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.533300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.533303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.533485 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:12.533928 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:12.533934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:12.533937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:12.534290 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:12.534669 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:12.535682 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0810 18:54:12.535973 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19465,"thread_start_us":25575,"threads_started":1}
I20260810 18:54:12.540719 3894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [term 1 FOLLOWER]: Refusing update from remote peer 5addaeb8b6e74c12aefe5b660b5d5d9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:12.541648 3972 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:12.542635 3800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: No bootstrap required, opened a new log
I20260810 18:54:12.559422 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5addaeb8b6e74c12aefe5b660b5d5d9d. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.562757 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.566061 3800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.566489 3800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e74dbaad8784c08b3db8181c907f6bf, State: Initialized, Role: FOLLOWER
I20260810 18:54:12.566768 3972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.567029 3972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:12.566973 3800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } }
I20260810 18:54:12.567755 4010 mvcc.cc:205] Tried to move back new op lower bound from 7317045463191146496 to 7317045462894682112. Current Snapshot: MvccSnapshot[applied={T|T < 7317045463191146496}]
I20260810 18:54:12.568540 3994 catalog_manager.cc:1347] Generated new cluster ID: 7716452d4ff54c2382e57cb6652c1495
I20260810 18:54:12.568651 3994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:12.571528 4017 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:54:12.578679 3945 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.578905 3945 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.578501 4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.580440 3748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4e74dbaad8784c08b3db8181c907f6bf"
I20260810 18:54:12.580793 3745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:12.580361 3800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:12.582490 4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.584551 3745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [term 1 FOLLOWER]: Refusing update from remote peer 5addaeb8b6e74c12aefe5b660b5d5d9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:12.585398 3983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:12.587759 3748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5addaeb8b6e74c12aefe5b660b5d5d9d in term 1.
I20260810 18:54:12.575809 3528 master_runner.cc:186] Error getting master registration for 127.29.174.253:36217: OK, instance_id { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" instance_seqno: 1786388051769650 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:54:12.593964 3528 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:54:12.609022 3994 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:12.610937 3994 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:12.623191 4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5addaeb8b6e74c12aefe5b660b5d5d9d. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.623437 4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.623986 4027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.624410 4027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:12.624516 3972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5addaeb8b6e74c12aefe5b660b5d5d9d" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 45503 } } peers { permanent_uuid: "4e74dbaad8784c08b3db8181c907f6bf" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 36217 } } peers { permanent_uuid: "a5e9c427117346cdaf517927bbcdd139" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 44503 } } }
I20260810 18:54:12.624761 3972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:12.635386 3994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5addaeb8b6e74c12aefe5b660b5d5d9d: Generated new TSK 0
I20260810 18:54:12.636543 3994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:12.717948 4057 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260810 18:54:12.741930 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.733148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.29.174.253:36217)
0810 18:54:12.733485 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.733491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.739437 (+ 5946us) server_negotiation.cc:407] Connection header received
0810 18:54:12.740377 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.740383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.740462 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.740534 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.741707 (+ 1173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45806: BlockingRecv error: recv got EOF from 127.0.0.1:45806 (error 108)
Metrics: {"server-negotiator.queue_time_us":209}
W20260810 18:54:12.745887 3943 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.732982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.29.174.254:45503)
0810 18:54:12.733331 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.733338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.743562 (+ 10224us) server_negotiation.cc:407] Connection header received
0810 18:54:12.743727 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.743733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.743808 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.743899 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.745642 (+ 1743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38500: BlockingRecv error: recv got EOF from 127.0.0.1:38500 (error 108)
Metrics: {"server-negotiator.queue_time_us":222}
W20260810 18:54:12.747303 3940 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.739890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.29.174.252:44503)
0810 18:54:12.740055 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.740061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.744350 (+ 4289us) server_negotiation.cc:407] Connection header received
0810 18:54:12.744570 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.744576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.744662 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.744734 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.747082 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53344: BlockingRecv error: recv got EOF from 127.0.0.1:53344 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 18:54:12.762079 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.760208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45812 (local address 127.29.174.253:36217)
0810 18:54:12.760373 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.760378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.760523 (+ 145us) server_negotiation.cc:407] Connection header received
0810 18:54:12.760819 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.760825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.760944 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.761048 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.761829 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45812: BlockingRecv error: recv got EOF from 127.0.0.1:45812 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:54:12.784718 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.772020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.29.174.253:36217)
0810 18:54:12.772174 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.772180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.783116 (+ 10936us) server_negotiation.cc:407] Connection header received
0810 18:54:12.783352 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.783366 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.783459 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.783529 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.784494 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45824: BlockingRecv error: recv got EOF from 127.0.0.1:45824 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 18:54:12.814478 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.812267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.29.174.253:36217)
0810 18:54:12.812452 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.812458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.812955 (+ 497us) server_negotiation.cc:407] Connection header received
0810 18:54:12.813148 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.813154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.813266 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.813384 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.814202 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45838: BlockingRecv error: recv got EOF from 127.0.0.1:45838 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 18:54:12.824831 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.821868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.29.174.253:36217)
0810 18:54:12.822099 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.822105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.823030 (+ 925us) server_negotiation.cc:407] Connection header received
0810 18:54:12.823417 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.823424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.823525 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.823651 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.824577 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45842: BlockingRecv error: recv got EOF from 127.0.0.1:45842 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260810 18:54:12.838833 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.836489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.29.174.253:36217)
0810 18:54:12.836684 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.836690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.837239 (+ 549us) server_negotiation.cc:407] Connection header received
0810 18:54:12.837438 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.837444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.837534 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.837637 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.838593 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45852: BlockingRecv error: recv got EOF from 127.0.0.1:45852 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 18:54:12.860708 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.858533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45860 (local address 127.29.174.253:36217)
0810 18:54:12.858719 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.858725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.859214 (+ 489us) server_negotiation.cc:407] Connection header received
0810 18:54:12.859413 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.859419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.859526 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.859637 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.860465 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45860: BlockingRecv error: recv got EOF from 127.0.0.1:45860 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 18:54:12.897769 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.895935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.29.174.253:36217)
0810 18:54:12.896151 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.896157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.896430 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:54:12.896585 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.896591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.896687 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.896816 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.897493 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45866: BlockingRecv error: recv got EOF from 127.0.0.1:45866 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260810 18:54:12.971614 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.969832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.29.174.253:36217)
0810 18:54:12.970039 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.970045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.970305 (+ 260us) server_negotiation.cc:407] Connection header received
0810 18:54:12.970502 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.970508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.970605 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.970731 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.971396 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45868: BlockingRecv error: recv got EOF from 127.0.0.1:45868 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 18:54:13.104434 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:13.102701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.29.174.253:36217)
0810 18:54:13.102919 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.102925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.103225 (+ 300us) server_negotiation.cc:407] Connection header received
0810 18:54:13.103377 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.103382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.103456 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.103541 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:13.104210 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45876: BlockingRecv error: recv got EOF from 127.0.0.1:45876 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:54:13.363517 3966 catalog_manager.cc:1259] Loaded cluster ID: 7716452d4ff54c2382e57cb6652c1495
I20260810 18:54:13.363653 3966 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: loading cluster ID for follower catalog manager: success
I20260810 18:54:13.366994 3966 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: acquiring CA information for follower catalog manager: success
W20260810 18:54:13.367755 3805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:13.365745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45880 (local address 127.29.174.253:36217)
0810 18:54:13.365972 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.365978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.366285 (+ 307us) server_negotiation.cc:407] Connection header received
0810 18:54:13.366531 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.366535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.366617 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.366709 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:13.367540 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45880: BlockingRecv error: recv got EOF from 127.0.0.1:45880 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:54:13.369117 3966 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5e9c427117346cdaf517927bbcdd139: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:54:13.722434 4057 catalog_manager.cc:1259] Loaded cluster ID: 7716452d4ff54c2382e57cb6652c1495
I20260810 18:54:13.722554 4057 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: loading cluster ID for follower catalog manager: success
I20260810 18:54:13.726220 4057 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: acquiring CA information for follower catalog manager: success
I20260810 18:54:13.728386 4057 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e74dbaad8784c08b3db8181c907f6bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:54:13.903730 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.896614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.29.174.253:36217)
0810 18:54:13.897286 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.897292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.897603 (+ 311us) server_negotiation.cc:407] Connection header received
0810 18:54:13.897821 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.897826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.897958 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.898057 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:13.898611 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.899402 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.901335 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.901797 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.901888 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.902529 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:13.903039 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:13.903142 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:13.903308 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 18:54:13.903379 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":350,"threads_started":1}
I20260810 18:54:13.918494 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.909897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.29.174.252:44503)
0810 18:54:13.913220 (+ 3323us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.913225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.913242 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:54:13.913324 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.913328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.913457 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.913581 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:13.914022 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.914756 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.916511 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.916890 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.916971 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.917494 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:13.917923 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:13.918024 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:13.918145 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 18:54:13.918205 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3230,"thread_start_us":3135,"threads_started":1}
I20260810 18:54:13.935101 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.926822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.29.174.253:36217)
0810 18:54:13.926999 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.927009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.927464 (+ 455us) server_negotiation.cc:407] Connection header received
0810 18:54:13.927772 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.927782 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.927884 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.927949 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:13.928866 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.929934 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.932027 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.932673 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.932814 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.933332 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:13.933473 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:13.933557 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:13.934675 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0810 18:54:13.934759 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:54:13.935447 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.927198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.29.174.254:45503)
0810 18:54:13.927748 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.927753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.928216 (+ 463us) server_negotiation.cc:407] Connection header received
0810 18:54:13.928518 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.928524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.928719 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.928871 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:13.929688 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.930770 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.932836 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.933430 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.933539 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.934330 (+ 791us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:13.934761 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:13.934834 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:13.935094 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 18:54:13.935177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
I20260810 18:54:13.940480 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.928454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.29.174.252:44503)
0810 18:54:13.928602 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.928607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.934237 (+ 5630us) server_negotiation.cc:407] Connection header received
0810 18:54:13.934305 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.934310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.934429 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.934509 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:13.935683 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.936518 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.938093 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.938660 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.938789 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.939325 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:13.939478 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:13.939580 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:13.940100 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 18:54:13.940175 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:54:13.941674 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3528
I20260810 18:54:13.942708 3654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:14.052387 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3528
I20260810 18:54:14.077457 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3670
I20260810 18:54:14.078529 3792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:14.181877 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3670
I20260810 18:54:14.204313 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 3808
I20260810 18:54:14.205355 3926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:14.308717 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 3808
2026-08-10T18:54:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4969 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:14 dist-test-slave-6zbq krb5kdc[4177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:14 dist-test-slave-6zbq krb5kdc[4177](info): set up 2 sockets
Aug 10 18:54:14 dist-test-slave-6zbq krb5kdc[4177](info): commencing operation
krb5kdc: starting...
W20260810 18:54:15.558992 30395 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.007s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/test-user.keytab.
Aug 10 18:54:15 dist-test-slave-6zbq krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.174.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
Entry for principal kudu/127.29.174.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab.
I20260810 18:54:15.822650 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39033
--webserver_interface=127.29.174.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:16.137835 4198 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:16.138103 4198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.138173 4198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.138257 4198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:16.145103 4198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:16.145176 4198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.145238 4198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.145287 4198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:16.145347 4198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:16.154408 4198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.254.keytab
--principal=kudu/127.29.174.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.254:39033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.174.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:16.155458 4198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.156579 4198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.167039 4217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.167135 4218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.167200 4223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.168107 4198 server_base.cc:1034] running on GCE node
I20260810 18:54:16.168761 4198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.169914 4198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.171118 4198 hybrid_clock.cc:648] HybridClock initialized: now 1786388056171063 us; error 27 us; skew 500 ppm
I20260810 18:54:16.171411 4198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:16.174288 4198 init.cc:377] Logged in from keytab as kudu/127.29.174.254@KRBTEST.COM (short username kudu)
I20260810 18:54:16.175637 4198 webserver.cc:460] Webserver started at http://127.29.174.254:35501/ using document root <none> and password file <none>
I20260810 18:54:16.176240 4198 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.176334 4198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.176625 4198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.179353 4198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data/instance:
uuid: "e0b32867884845ff81e4064ed0cefc8c"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "c0f1ee69befddbd34dadab49e952c692"
server_key_iv: "ecd86c322984ceed681a1b4737243885"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.180075 4198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance:
uuid: "e0b32867884845ff81e4064ed0cefc8c"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "c0f1ee69befddbd34dadab49e952c692"
server_key_iv: "ecd86c322984ceed681a1b4737243885"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.184777 4198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:54:16.187978 4234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.189141 4198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:54:16.189294 4198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal
uuid: "e0b32867884845ff81e4064ed0cefc8c"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "c0f1ee69befddbd34dadab49e952c692"
server_key_iv: "ecd86c322984ceed681a1b4737243885"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.189478 4198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.205062 4198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.208549 4198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.208772 4198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.225131 4198 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.254:39033
I20260810 18:54:16.225183 4313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.254:39033 every 8 connection(s)
I20260810 18:54:16.226300 4198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:16.229754 4314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:16.234462 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4198
I20260810 18:54:16.234655 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-0/wal/instance
I20260810 18:54:16.235178 30395 external_mini_cluster.cc:1428] Setting key eadbc44394d7f1f967878163c378ecb8
I20260810 18:54:16.236603 4314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:16.262323 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.240986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.29.174.254:39033)
0810 18:54:16.241658 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.241667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.241715 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:54:16.242962 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.242969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.243234 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.243438 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.244475 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.246041 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.247404 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.248129 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.248289 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.255871 (+ 7582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.255879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.255889 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:16.255920 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.258515 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.259135 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.259141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.259144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.259282 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.259639 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.259649 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.259651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.260071 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.260255 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.260749 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.261170 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":463,"thread_start_us":324,"threads_started":1}
I20260810 18:54:16.262375 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.239837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39033 (local address 127.0.0.1:33242)
0810 18:54:16.240590 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.240634 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.242622 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.243437 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.243442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.243953 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.244276 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.244292 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.246201 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.246206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.247266 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.247283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.248318 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.248323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.248458 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.253299 (+ 4841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:16.253332 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.255671 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.258636 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.258648 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.258658 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.259012 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.259382 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.259386 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.259390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.259533 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.260204 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.260211 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.260392 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.260587 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.261017 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":315,"threads_started":1}
WARNING: no policy specified for kudu/127.29.174.253@KRBTEST.COM; defaulting to no policy
I20260810 18:54:16.265334 4314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:16.266381 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.266000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56804)
0810 18:54:16.266143 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.266246 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:54:16.266633 4242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.253@KRBTEST.COM" created.
I20260810 18:54:16.268028 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.267654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56806)
0810 18:54:16.267811 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.267902 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 18:54:16.268204 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:45231: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
Entry for principal kudu/127.29.174.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab.
I20260810 18:54:16.298089 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:45231
--webserver_interface=127.29.174.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:54:16.317538 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } attempt: 1
I20260810 18:54:16.318811 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.318262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56822)
0810 18:54:16.318502 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.318631 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 18:54:16.319810 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.319409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56824)
0810 18:54:16.319563 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.319652 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:54:16.320021 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:45231: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
I20260810 18:54:16.388288 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } attempt: 2
I20260810 18:54:16.389492 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.388947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56828)
0810 18:54:16.389212 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.389323 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:54:16.390648 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.390165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56838)
0810 18:54:16.390315 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.390453 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:54:16.390846 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:45231: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
I20260810 18:54:16.482126 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } attempt: 3
I20260810 18:54:16.483436 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.482885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56854)
0810 18:54:16.483135 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.483258 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 18:54:16.484553 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.484120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56862)
0810 18:54:16.484280 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.484374 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:54:16.484766 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:45231: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
W20260810 18:54:16.621728 4327 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:16.622073 4327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.622149 4327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.622236 4327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:54:16.628019 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } attempt: 4
I20260810 18:54:16.629271 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.628691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56870)
0810 18:54:16.628998 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.629102 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260810 18:54:16.629793 4327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:16.629873 4327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.629958 4327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.630030 4327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:16.630117 4327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:16.630375 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.629944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56876)
0810 18:54:16.630100 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.630203 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:54:16.630594 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.253:45231: Network error: Client connection negotiation failed: client connection to 127.29.174.253:45231: connect: Connection refused (error 111)
I20260810 18:54:16.640137 4327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.253.keytab
--principal=kudu/127.29.174.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.253:45231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.174.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:16.641388 4327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.642730 4327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.653810 4334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.653988 4337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.654065 4335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.654867 4327 server_base.cc:1034] running on GCE node
I20260810 18:54:16.655603 4327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.656957 4327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.658171 4327 hybrid_clock.cc:648] HybridClock initialized: now 1786388056658110 us; error 37 us; skew 500 ppm
I20260810 18:54:16.658517 4327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:16.661473 4327 init.cc:377] Logged in from keytab as kudu/127.29.174.253@KRBTEST.COM (short username kudu)
I20260810 18:54:16.662811 4327 webserver.cc:460] Webserver started at http://127.29.174.253:40813/ using document root <none> and password file <none>
I20260810 18:54:16.663421 4327 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.663506 4327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.663772 4327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.666558 4327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data/instance:
uuid: "940393d075694102b1e785d3c07ceb5e"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "cea020dd4ddf67ba6c5bc900e48fc654"
server_key_iv: "d32b8f0b3c4d9e19bf9d42c1b7c24616"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.667294 4327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal/instance:
uuid: "940393d075694102b1e785d3c07ceb5e"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "cea020dd4ddf67ba6c5bc900e48fc654"
server_key_iv: "d32b8f0b3c4d9e19bf9d42c1b7c24616"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.672037 4327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:54:16.675206 4347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.676383 4327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:54:16.676537 4327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal
uuid: "940393d075694102b1e785d3c07ceb5e"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-6zbq"
server_key: "cea020dd4ddf67ba6c5bc900e48fc654"
server_key_iv: "d32b8f0b3c4d9e19bf9d42c1b7c24616"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.676718 4327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.705717 4327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.709549 4327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.709782 4327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.725910 4327 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.253:45231
I20260810 18:54:16.725932 4438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.253:45231 every 8 connection(s)
I20260810 18:54:16.727106 4327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/data/info.pb
I20260810 18:54:16.730531 4439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:16.733618 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4327
I20260810 18:54:16.733819 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-1/wal/instance
I20260810 18:54:16.734408 30395 external_mini_cluster.cc:1428] Setting key e48a0af767f54d904671e32acea5ec7e
I20260810 18:54:16.738104 4439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:16.757088 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.740947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.29.174.254:39033)
0810 18:54:16.741157 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.741164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.742157 (+ 993us) server_negotiation.cc:407] Connection header received
0810 18:54:16.743990 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.743997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.744076 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.744194 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.745502 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.746453 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.747833 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.748357 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.748473 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.751644 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.751650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.751653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:16.751686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.754186 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.754907 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.754912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.754915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.755036 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.755437 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.755440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.755443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.755732 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.755865 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.756536 (+ 671us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.756774 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:54:16.757916 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.741277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39033 (local address 127.0.0.1:33250)
0810 18:54:16.742002 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.742043 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.743730 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.744354 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.744361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.744847 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.745337 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.745354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.746580 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.746584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.747685 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.747697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.748528 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.748532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.748719 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.749297 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:16.749318 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.751431 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.754334 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.754363 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.754377 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.754790 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.755142 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.755147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.755150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.755317 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.755858 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.755865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.756178 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.756396 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.756846 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":297,"threads_started":1}
I20260810 18:54:16.759486 4439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.174.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.174.252@KRBTEST.COM" created.
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:16.774847 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.760179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56878)
0810 18:54:16.760350 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.760367 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.760466 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.761445 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.761449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.761901 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.762085 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.762097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.763530 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.763536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.764871 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.764887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.765669 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.765679 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.765870 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.766409 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:16.766425 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.769709 (+ 3284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.772513 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.772517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.772521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.772959 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.773281 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.773285 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.773288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.773443 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.773971 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.773976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.774103 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.774239 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.774511 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:54:16.774926 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.760412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.29.174.253:45231)
0810 18:54:16.761036 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.761041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.761065 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:54:16.761122 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.761127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.761287 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.761396 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.762308 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.763388 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.765006 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.765514 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.765604 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.769917 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.769928 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.769940 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:16.769968 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.772387 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.773068 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.773073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.773075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.773184 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.773548 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.773551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.773553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.773855 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.774011 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.774380 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.774629 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":480,"thread_start_us":353,"threads_started":1}
I20260810 18:54:16.777324 4439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:16.778328 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.777859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:40990)
0810 18:54:16.778019 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.778166 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:54:16.778589 4354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:16.780061 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.779710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:40994)
0810 18:54:16.779865 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.779945 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:54:16.780334 4439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
Entry for principal kudu/127.29.174.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab.
I20260810 18:54:16.802122 30395 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
/tmp/dist-test-taskY4kHxe/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:39545
--webserver_interface=127.29.174.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.174.212:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:54:16.829631 4439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 1
I20260810 18:54:16.830706 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.830270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:40998)
0810 18:54:16.830460 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.830556 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:54:16.831708 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.831321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41008)
0810 18:54:16.831477 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.831568 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 18:54:16.831920 4439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:16.900148 4439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 2
I20260810 18:54:16.901346 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.900804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41024)
0810 18:54:16.901019 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.901147 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:54:16.902305 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.901951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41028)
0810 18:54:16.902087 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.902171 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:54:16.902527 4439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:16.929821 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } attempt: 5
Aug 10 18:54:16 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:16.942456 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.930559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56894)
0810 18:54:16.930792 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.930814 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.930949 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.931312 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.931316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.931703 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.931891 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.931901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.932919 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.932925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.933779 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.933792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.934531 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.934536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.934758 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.935319 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:16.935336 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.937247 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.940173 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.940178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.940182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.940539 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.940878 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:16.940883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:16.940887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:16.941016 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:16.941513 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.941518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.941668 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.941786 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.942050 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 18:54:16.942459 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.930657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56894 (local address 127.29.174.253:45231)
0810 18:54:16.930840 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.930846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.930874 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:54:16.931091 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.931095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.931161 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.931245 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.932041 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.932793 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.933925 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.934396 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.934509 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.937420 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.937425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.937428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:16.937460 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.940046 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.940654 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.940659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.940661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.940778 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:16.941126 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:16.941131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:16.941135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:16.941403 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.941534 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.941910 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.942118 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 18:54:16.943720 4314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:16.944677 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.944321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41044)
0810 18:54:16.944470 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.944541 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:54:16.945549 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.945241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41050)
0810 18:54:16.945375 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.945448 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:54:16.945744 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:16.993743 4439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 3
I20260810 18:54:16.995034 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.994448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41056)
0810 18:54:16.994688 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.994817 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:54:16.996280 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.995768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41064)
0810 18:54:16.996020 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.996122 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260810 18:54:16.996512 4439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:16.996930 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 1
I20260810 18:54:16.997777 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.997393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41078)
0810 18:54:16.997558 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.997635 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:54:16.998688 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.998416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41088)
0810 18:54:16.998530 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.998595 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 18:54:16.998873 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:17.067060 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 2
I20260810 18:54:17.068219 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.067743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41096)
0810 18:54:17.067926 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.068039 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:54:17.069216 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.068869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41106)
0810 18:54:17.069019 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.069115 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:54:17.069425 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
W20260810 18:54:17.121165 4452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:54:17.121446 4452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:17.121546 4452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:17.121649 4452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:17.128546 4452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:17.128623 4452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:17.128712 4452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:17.128783 4452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:17.128867 4452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:17.138271 4452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.174.212:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.174.254:39033,127.29.174.253:45231,127.29.174.252:39545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/krb5kdc/kudu_127.29.174.252.keytab
--principal=kudu/127.29.174.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.174.252:39545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.174.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:54:17.139599 4452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:17.139757 4439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 4
I20260810 18:54:17.140903 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.140359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41118)
0810 18:54:17.140620 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.140723 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 18:54:17.141242 4452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:54:17.141892 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.141534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41128)
0810 18:54:17.141681 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.141766 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:54:17.142093 4439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
W20260810 18:54:17.152391 4461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.152441 4464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.152478 4459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:17.153281 4452 server_base.cc:1034] running on GCE node
I20260810 18:54:17.153975 4452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:17.155256 4452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:17.156494 4452 hybrid_clock.cc:648] HybridClock initialized: now 1786388057156429 us; error 43 us; skew 500 ppm
I20260810 18:54:17.156787 4452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388057, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:17.159730 4452 init.cc:377] Logged in from keytab as kudu/127.29.174.252@KRBTEST.COM (short username kudu)
I20260810 18:54:17.160961 4452 webserver.cc:460] Webserver started at http://127.29.174.252:45597/ using document root <none> and password file <none>
I20260810 18:54:17.161597 4452 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:17.161696 4452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:17.161986 4452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:17.164770 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data/instance:
uuid: "298991c471014ceaac4be79a9db70298"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-6zbq"
server_key: "cdf2bb8ce7281643b28353de4ed390a3"
server_key_iv: "5b0a3e4ad7e48836c3fa86d0d75bcd42"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.165521 4452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal/instance:
uuid: "298991c471014ceaac4be79a9db70298"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-6zbq"
server_key: "cdf2bb8ce7281643b28353de4ed390a3"
server_key_iv: "5b0a3e4ad7e48836c3fa86d0d75bcd42"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.170045 4452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 18:54:17.173130 4477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.174250 4452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:54:17.174453 4452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data,/tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal
uuid: "298991c471014ceaac4be79a9db70298"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-6zbq"
server_key: "cdf2bb8ce7281643b28353de4ed390a3"
server_key_iv: "5b0a3e4ad7e48836c3fa86d0d75bcd42"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.174635 4452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:17.175631 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 3
I20260810 18:54:17.176683 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.176161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41130)
0810 18:54:17.176389 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.176491 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:54:17.177629 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.177324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41142)
0810 18:54:17.177464 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.177528 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 18:54:17.177840 4314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.174.252:39545: Network error: Client connection negotiation failed: client connection to 127.29.174.252:39545: connect: Connection refused (error 111)
I20260810 18:54:17.189359 4452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:17.192835 4452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:17.193068 4452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:17.209542 4452 rpc_server.cc:307] RPC server started. Bound to: 127.29.174.252:39545
I20260810 18:54:17.209560 4554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.174.252:39545 every 8 connection(s)
I20260810 18:54:17.210752 4452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/data/info.pb
I20260810 18:54:17.214126 4556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:17.216462 30395 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu as pid 4452
I20260810 18:54:17.216655 30395 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY4kHxe/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387695213813-30395-0/minicluster-data/master-2/wal/instance
I20260810 18:54:17.217162 30395 external_mini_cluster.cc:1428] Setting key e7d891a6cd023c6998a979f464f9ba89
I20260810 18:54:17.221449 4556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388057, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:17.239708 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.224130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.254:39033 (local address 127.0.0.1:33266)
0810 18:54:17.224962 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.225001 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.225281 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.225745 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.225753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.226663 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.226977 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.226989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.228314 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.228319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.229769 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.229786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.230844 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.230852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.231031 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.231690 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:17.231719 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.233939 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.237173 (+ 3234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.237181 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.237186 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.237600 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.238006 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.238010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.238013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.238172 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.238792 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.238797 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.238948 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.239086 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.239297 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"spinlock_wait_cycles":12032,"thread_start_us":355,"threads_started":1}
I20260810 18:54:17.239768 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.224141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.29.174.254:39033)
0810 18:54:17.224353 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.224360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.225079 (+ 719us) server_negotiation.cc:407] Connection header received
0810 18:54:17.225455 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.225465 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.225547 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.225661 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.227121 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.228155 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.229929 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.230619 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.230735 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.234130 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.234137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.234142 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.234181 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.237063 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.237713 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.237718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.237721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.237884 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.238302 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.238307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.238310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.238668 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.238822 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.239220 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.239449 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:54:17.239768 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.218816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.29.174.252:39545)
0810 18:54:17.219561 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.219571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.219613 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:54:17.221193 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.221214 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.221567 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.221855 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.223587 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.225349 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.227167 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.227752 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.227922 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.232494 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.232505 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.232518 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.232570 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.235955 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.236581 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.236590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.236597 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.236791 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.237114 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.237119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.237122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.237617 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.237928 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.238188 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.238580 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":328,"threads_started":1}
I20260810 18:54:17.241439 4556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388057, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.254@KRBTEST.COM
I20260810 18:54:17.254312 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.242133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.253:45231 (local address 127.0.0.1:56910)
0810 18:54:17.242471 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.242487 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.242581 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.242906 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.242910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.243204 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.243333 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.243340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.244403 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.244408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.245340 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.245352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.246167 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.246173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.246381 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.246902 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:17.246917 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.248755 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.251890 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.251896 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.251900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.252355 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.252670 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.252675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.252679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.252844 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.253435 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.253440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.253622 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.253790 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.254008 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248}
I20260810 18:54:17.254395 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.242126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.29.174.253:45231)
0810 18:54:17.242303 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.242308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.242566 (+ 258us) server_negotiation.cc:407] Connection header received
0810 18:54:17.242686 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.242691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.242771 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.242870 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.243472 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.244270 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.245474 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.246038 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.246127 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.248949 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.248956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.248961 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.248999 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.251769 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.252455 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.252458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.252460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.252574 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.252952 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.252956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.252958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.253294 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.253443 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.253897 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.254106 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:54:17.255362 4556 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:54:17.255677 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.242135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33272 (local address 127.29.174.254:39033)
0810 18:54:17.242412 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.242417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.242434 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:54:17.242488 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.242492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.242558 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.242649 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.243824 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.244573 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.245810 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.246235 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.246429 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.250181 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.250186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.250189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.250217 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.252844 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.253572 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.253577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.253581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.253735 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.254121 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.254126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.254130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.254702 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.254937 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.255240 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.255424 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388057, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.252@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.174.253@KRBTEST.COM
I20260810 18:54:17.268178 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.256009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41170)
0810 18:54:17.256147 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.256164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.256252 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.256494 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.256497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.256877 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.256999 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.257007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.257963 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.257968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.259245 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.259257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.260084 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.260088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.260304 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.260786 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:17.260808 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.263173 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.265972 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.265978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.265982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.266422 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.266754 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.266757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.266759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.266871 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.267348 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.267352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.267471 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.267581 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.267837 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 18:54:17.268321 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.256123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.29.174.252:39545)
0810 18:54:17.256257 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.256262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.256280 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:54:17.256323 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.256327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.256391 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.256491 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.257133 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.257846 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.259365 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.259960 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.260080 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.263342 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.263347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.263350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.263378 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.265846 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.266523 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.266529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.266532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.266663 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.266976 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.266980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.266982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.267241 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.267361 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.267692 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.267989 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":46}
I20260810 18:54:17.271478 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.257276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.29.174.253:45231)
0810 18:54:17.257414 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.257419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.257435 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:54:17.257582 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.257587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.257639 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.257721 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.259154 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.260094 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.261727 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.262236 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.262380 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.266288 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.266292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.266295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.266322 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.269095 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.269723 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.269728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.269731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.269888 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.270221 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.270225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.270228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.270569 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.270839 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.270970 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.271175 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:54:17.273970 4556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: Bootstrap starting.
I20260810 18:54:17.275731 30395 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:54:17.277433 4556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:17.278751 4556 log.cc:824] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:17.277992 4277 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e0b32867884845ff81e4064ed0cefc8c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33272
I20260810 18:54:17.281786 4556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: No bootstrap required, opened a new log
I20260810 18:54:17.292004 4556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.292382 4556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298991c471014ceaac4be79a9db70298, State: Initialized, Role: FOLLOWER
I20260810 18:54:17.292829 4556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.293681 4575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.294028 4575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.294808 4556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:54:17.303828 4600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:54:17.303959 4600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:54:17.301481 4452 master_runner.cc:186] Error getting master registration for 127.29.174.254:39033: OK, instance_id { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" instance_seqno: 1786388056220496 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:54:17.311973 4452 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:54:17.355038 4314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 4
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.254@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:54:17.369753 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.355716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41178)
0810 18:54:17.355890 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.355911 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.356011 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.356343 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.356348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.356789 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.356965 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.356975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.358045 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.358051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.359162 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.359179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.360140 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.360146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.360355 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.361000 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:17.361025 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.363404 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.367119 (+ 3715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.367123 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.367126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.367616 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.367990 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.367995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.367999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.368158 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.368820 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.368824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.368977 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.369119 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.369374 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:54:17.369835 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.355794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.29.174.252:39545)
0810 18:54:17.355989 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.355995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.356021 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:54:17.356137 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.356143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.356207 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.356285 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.357106 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.357903 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.359321 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.359997 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.360135 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.363586 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.363593 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.363598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.363637 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.366973 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.367733 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.367738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.367742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.367885 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.368271 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.368276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.368280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.368687 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.368862 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.369246 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.369479 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:54:17.375458 4314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: Bootstrap starting.
I20260810 18:54:17.379328 4314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:17.380620 4314 log.cc:824] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:17.383710 4314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: No bootstrap required, opened a new log
I20260810 18:54:17.390974 4314 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.391355 4314 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b32867884845ff81e4064ed0cefc8c, State: Initialized, Role: FOLLOWER
I20260810 18:54:17.391824 4314 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.392786 4605 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.393159 4605 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.393951 4314 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:17.394083 4277 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:54:17.394284 4277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:17.396953 4277 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.398124 4277 leader_election.cc:288] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [CANDIDATE]: Term 1 election: Requested vote from peers 940393d075694102b1e785d3c07ceb5e (127.29.174.253:45231), 298991c471014ceaac4be79a9db70298 (127.29.174.252:39545)
I20260810 18:54:17.399250 4519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0b32867884845ff81e4064ed0cefc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "298991c471014ceaac4be79a9db70298"
I20260810 18:54:17.399711 4519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:17.403240 4519 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0b32867884845ff81e4064ed0cefc8c in term 1.
I20260810 18:54:17.403805 4242 leader_election.cc:302] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 298991c471014ceaac4be79a9db70298, e0b32867884845ff81e4064ed0cefc8c; no voters:
I20260810 18:54:17.404191 4605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:17.404764 4605 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [term 1 LEADER]: Becoming Leader. State: Replica: e0b32867884845ff81e4064ed0cefc8c, State: Running, Role: LEADER
I20260810 18:54:17.405251 4605 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.422098 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b32867884845ff81e4064ed0cefc8c. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.422520 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:17.430459 4625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:17.441290 4439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } attempt: 5
I20260810 18:54:17.441485 4625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 18:54:17.444897 4489 tablet.cc:2369] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 18:54:17 dist-test-slave-6zbq krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388056, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.174.253@KRBTEST.COM for kudu/127.29.174.252@KRBTEST.COM
I20260810 18:54:17.458472 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.441840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.174.252:39545 (local address 127.0.0.1:41186)
0810 18:54:17.442205 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.442223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.442358 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.443288 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.443294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.443777 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.443977 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.443990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.445435 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.445441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.446836 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:17.446853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.447950 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.447956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:17.448182 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.448821 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:17.448843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.451816 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.455689 (+ 3873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.455694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.455699 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.456165 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.456610 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:17.456615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:17.456619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:17.456819 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:17.457504 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.457509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.457656 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.457791 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.458052 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260810 18:54:17.459362 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.442681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.29.174.252:39545)
0810 18:54:17.442846 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.442853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.442871 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:54:17.442919 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.442924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.442999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.443094 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.444160 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.445269 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.447016 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.447647 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.447761 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.452022 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.452029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.452034 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.452073 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.455530 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.456320 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.456326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.456330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.456474 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.456958 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.456963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.456966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.457296 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.457436 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.458804 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.459002 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:54:17.462208 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.430851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.29.174.254:39033)
0810 18:54:17.431015 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.431021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.434926 (+ 3905us) server_negotiation.cc:407] Connection header received
0810 18:54:17.435207 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.435214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.435291 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.435367 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.447597 (+ 12230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.448630 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.450401 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.451059 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.451176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.454946 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.454953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.454958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.454995 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.458329 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.459191 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.459196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.459200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.459359 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.459872 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.459877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.459880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.460251 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.460518 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.461579 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.461833 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 18:54:17.462458 4244 tablet.cc:2369] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:54:17.465229 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: Bootstrap starting.
I20260810 18:54:17.466379 4644 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:54:17.471091 4439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:17.473089 4439 log.cc:824] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:17.470752 4198 master_runner.cc:186] Error getting master registration for 127.29.174.253:45231: OK, instance_id { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" instance_seqno: 1786388056721311 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:54:17.481217 4198 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:54:17.488327 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: No bootstrap required, opened a new log
I20260810 18:54:17.490746 4519 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [term 1 FOLLOWER]: Refusing update from remote peer e0b32867884845ff81e4064ed0cefc8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:17.491799 4605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:17.494275 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.435634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.29.174.253:45231)
0810 18:54:17.435806 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.435812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.455957 (+ 20145us) server_negotiation.cc:407] Connection header received
0810 18:54:17.457147 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.457153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.457228 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.457292 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.461002 (+ 3710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.462136 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.470768 (+ 8632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.471402 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.471500 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.477247 (+ 5747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.477254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.477258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.477291 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.480774 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.491134 (+ 10360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.491140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.491143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.491334 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.492030 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.492035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.492038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.492423 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.492689 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.493603 (+ 914us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.493890 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:54:17.505606 4439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.506263 4439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940393d075694102b1e785d3c07ceb5e, State: Initialized, Role: FOLLOWER
I20260810 18:54:17.507105 4439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } }
I20260810 18:54:17.509861 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.447306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.29.174.252:39545)
0810 18:54:17.459582 (+ 12276us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.459587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.477094 (+ 17507us) server_negotiation.cc:407] Connection header received
0810 18:54:17.478262 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.478270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.478374 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.478460 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.482742 (+ 4282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.483812 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.486959 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.487531 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.487619 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.501188 (+ 13569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.501198 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.501203 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:17.501257 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.504938 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.506260 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.506273 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.506277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.506502 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:17.507346 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:17.507352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:17.507355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:17.507837 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.508111 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.509179 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.509456 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12174,"thread_start_us":31390,"threads_started":1}
I20260810 18:54:17.522948 4648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.523450 4648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.524442 4575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b32867884845ff81e4064ed0cefc8c. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.524820 4575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.525689 4396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0b32867884845ff81e4064ed0cefc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "940393d075694102b1e785d3c07ceb5e"
I20260810 18:54:17.526262 4439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:17.526460 4393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:17.530143 4393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [term 1 FOLLOWER]: Refusing update from remote peer e0b32867884845ff81e4064ed0cefc8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:17.533429 4396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0b32867884845ff81e4064ed0cefc8c in term 1.
I20260810 18:54:17.535933 4646 mvcc.cc:205] Tried to move back new op lower bound from 7317045483456835584 to 7317045483135840256. Current Snapshot: MvccSnapshot[applied={T|T < 7317045483456835584}]
I20260810 18:54:17.537549 4614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:17.551880 4575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.552145 4575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.553666 4614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.553954 4614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:17.560957 4625 catalog_manager.cc:1347] Generated new cluster ID: a40dce7a85ea4e34b8d031372b51be87
I20260810 18:54:17.561061 4625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:17.575021 4648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b32867884845ff81e4064ed0cefc8c. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.575374 4648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.575704 4661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.576046 4661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:54:17.595513 4605 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0b32867884845ff81e4064ed0cefc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0b32867884845ff81e4064ed0cefc8c" member_type: VOTER last_known_addr { host: "127.29.174.254" port: 39033 } } peers { permanent_uuid: "940393d075694102b1e785d3c07ceb5e" member_type: VOTER last_known_addr { host: "127.29.174.253" port: 45231 } } peers { permanent_uuid: "298991c471014ceaac4be79a9db70298" member_type: VOTER last_known_addr { host: "127.29.174.252" port: 39545 } } }
I20260810 18:54:17.595827 4605 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:17.603186 4625 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:17.606523 4625 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:17.621111 4625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0b32867884845ff81e4064ed0cefc8c: Generated new TSK 0
I20260810 18:54:17.622107 4625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:17.661208 4672 catalog_manager.cc:1259] Loaded cluster ID: a40dce7a85ea4e34b8d031372b51be87
I20260810 18:54:17.661403 4672 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: loading cluster ID for follower catalog manager: success
I20260810 18:54:17.664978 4672 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: acquiring CA information for follower catalog manager: success
I20260810 18:54:17.667321 4672 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 940393d075694102b1e785d3c07ceb5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:54:17.753427 4322 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.737857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.29.174.254:39033)
0810 18:54:17.738079 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.738086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.751342 (+ 13256us) server_negotiation.cc:407] Connection header received
0810 18:54:17.751851 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.751858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.751944 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.752020 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.753171 (+ 1151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33282: BlockingRecv error: recv got EOF from 127.0.0.1:33282 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 18:54:17.755868 4448 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.746369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.29.174.253:45231)
0810 18:54:17.746567 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.746573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.754381 (+ 7808us) server_negotiation.cc:407] Connection header received
0810 18:54:17.754665 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.754671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.754768 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.754835 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.755639 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56934: BlockingRecv error: recv got EOF from 127.0.0.1:56934 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 18:54:17.759215 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.756916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.29.174.252:39545)
0810 18:54:17.757101 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.757108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.757709 (+ 601us) server_negotiation.cc:407] Connection header received
0810 18:54:17.757941 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.757946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.758025 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.758097 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.758952 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41200: BlockingRecv error: recv got EOF from 127.0.0.1:41200 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:54:17.804395 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.789157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.29.174.253:45231)
0810 18:54:17.789342 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.789349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.796651 (+ 7302us) server_negotiation.cc:407] Connection header received
0810 18:54:17.797044 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.797050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.797301 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.797445 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.798299 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.799431 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.801427 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.802080 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.802180 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.802923 (+ 743us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.803581 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.803712 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.803976 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.804071 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 18:54:17.840408 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.836743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41206 (local address 127.29.174.252:39545)
0810 18:54:17.836927 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.836933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.838892 (+ 1959us) server_negotiation.cc:407] Connection header received
0810 18:54:17.839073 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.839079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.839201 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.839322 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.840158 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41206: BlockingRecv error: recv got EOF from 127.0.0.1:41206 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:54:17.852746 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.845177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.29.174.253:45231)
0810 18:54:17.845352 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.845358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.846819 (+ 1461us) server_negotiation.cc:407] Connection header received
0810 18:54:17.846986 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.846991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.847095 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.847185 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.847632 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.848767 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.850681 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.851382 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.851507 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.851959 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.852114 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.852218 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.852359 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.852445 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 18:54:17.862286 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.860188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.29.174.252:39545)
0810 18:54:17.860391 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.860396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.860901 (+ 505us) server_negotiation.cc:407] Connection header received
0810 18:54:17.861051 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.861056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.861166 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.861273 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.862065 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41222: BlockingRecv error: recv got EOF from 127.0.0.1:41222 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:54:17.873581 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.866991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.29.174.253:45231)
0810 18:54:17.867144 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.867149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.867652 (+ 503us) server_negotiation.cc:407] Connection header received
0810 18:54:17.867811 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.867817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.867902 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.867992 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.868405 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.869542 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.871507 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.872209 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.872360 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.872795 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.872935 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.873027 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.873169 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.873253 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 18:54:17.892321 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.888642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.29.174.252:39545)
0810 18:54:17.888829 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.888836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.890853 (+ 2017us) server_negotiation.cc:407] Connection header received
0810 18:54:17.891037 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.891043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.891137 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.891244 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.892072 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41236: BlockingRecv error: recv got EOF from 127.0.0.1:41236 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:54:17.912886 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.903596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.29.174.253:45231)
0810 18:54:17.903801 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.903807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.906848 (+ 3041us) server_negotiation.cc:407] Connection header received
0810 18:54:17.907028 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.907035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.907127 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.907244 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.907699 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.908868 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.910827 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.911533 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.911671 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.912137 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.912285 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.912386 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.912521 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.912606 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260810 18:54:17.924321 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.919458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.29.174.252:39545)
0810 18:54:17.919715 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.919721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.922912 (+ 3191us) server_negotiation.cc:407] Connection header received
0810 18:54:17.923080 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.923086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.923185 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.923292 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.924101 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41238: BlockingRecv error: recv got EOF from 127.0.0.1:41238 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
I20260810 18:54:17.942756 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.936128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.29.174.253:45231)
0810 18:54:17.936330 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.936336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.936849 (+ 513us) server_negotiation.cc:407] Connection header received
0810 18:54:17.937021 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.937027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.937105 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.937195 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.937660 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.938767 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.940676 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.941354 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.941476 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.941955 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.942090 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.942176 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.942370 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.942448 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 18:54:17.951180 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.948950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.29.174.252:39545)
0810 18:54:17.949109 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.949115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.949793 (+ 678us) server_negotiation.cc:407] Connection header received
0810 18:54:17.949958 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.949963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.950043 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.950136 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.950985 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41246: BlockingRecv error: recv got EOF from 127.0.0.1:41246 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 18:54:17.977144 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.970299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.29.174.253:45231)
0810 18:54:17.970507 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.970513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.971015 (+ 502us) server_negotiation.cc:407] Connection header received
0810 18:54:17.971196 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.971202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.971301 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.971396 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:17.971872 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.973046 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.975022 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:17.975707 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:17.975833 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:17.976334 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:17.976518 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:17.976614 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.976783 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.976880 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260810 18:54:17.989166 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.987112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41260 (local address 127.29.174.252:39545)
0810 18:54:17.987287 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.987293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.987804 (+ 511us) server_negotiation.cc:407] Connection header received
0810 18:54:17.987962 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.987968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.988054 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.988150 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.988933 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41260: BlockingRecv error: recv got EOF from 127.0.0.1:41260 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:54:18.030176 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.024416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.29.174.253:45231)
0810 18:54:18.024618 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.024624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.024940 (+ 316us) server_negotiation.cc:407] Connection header received
0810 18:54:18.025089 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.025095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.025209 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.025337 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.025700 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.026886 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.028259 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.028945 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.029092 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.029417 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.029547 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.029663 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.029761 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.029842 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 18:54:18.038064 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:18.036255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.29.174.252:39545)
0810 18:54:18.036437 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.036442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.036681 (+ 239us) server_negotiation.cc:407] Connection header received
0810 18:54:18.036849 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.036853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.036923 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.036999 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:18.037835 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41262: BlockingRecv error: recv got EOF from 127.0.0.1:41262 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:54:18.110875 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.105423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56992 (local address 127.29.174.253:45231)
0810 18:54:18.105613 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.105620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.105966 (+ 346us) server_negotiation.cc:407] Connection header received
0810 18:54:18.106133 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.106139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.106229 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.106333 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.106706 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.107786 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.109072 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.109708 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.109826 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.110140 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.110283 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.110406 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.110499 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.110577 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 18:54:18.119896 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:18.116677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.29.174.252:39545)
0810 18:54:18.116878 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.116884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.118760 (+ 1876us) server_negotiation.cc:407] Connection header received
0810 18:54:18.118896 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.118902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.118994 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.119082 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:18.119692 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41278: BlockingRecv error: recv got EOF from 127.0.0.1:41278 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:54:18.255983 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.251091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.29.174.253:45231)
0810 18:54:18.251287 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.251294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.251631 (+ 337us) server_negotiation.cc:407] Connection header received
0810 18:54:18.251821 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.251825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.251895 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.251977 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.252402 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.253076 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.254398 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.254814 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.254923 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.255336 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.255440 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.255514 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.255668 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.255746 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 18:54:18.262960 4565 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:18.261271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.29.174.252:39545)
0810 18:54:18.261466 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.261472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.261745 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:54:18.261875 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.261878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.261944 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.262020 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:18.262761 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41292: BlockingRecv error: recv got EOF from 127.0.0.1:41292 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:54:18.305800 4600 catalog_manager.cc:1259] Loaded cluster ID: a40dce7a85ea4e34b8d031372b51be87
I20260810 18:54:18.305884 4600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: loading cluster ID for follower catalog manager: success
I20260810 18:54:18.308357 4600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: acquiring CA information for follower catalog manager: success
I20260810 18:54:18.310062 4600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 298991c471014ceaac4be79a9db70298: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:54:18.529469 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.524364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.29.174.253:45231)
0810 18:54:18.524769 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.524775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.524840 (+ 65us) server_negotiation.cc:407] Connection header received
0810 18:54:18.525083 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.525090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.525183 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.525295 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.525877 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.526796 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.528118 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.528513 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.528606 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.528907 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.529002 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.529065 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.529142 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.529200 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260810 18:54:18.539461 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.534679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41306 (local address 127.29.174.252:39545)
0810 18:54:18.534888 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.534893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.534992 (+ 99us) server_negotiation.cc:407] Connection header received
0810 18:54:18.535120 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.535124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.535277 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.535369 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.535676 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.536372 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.537521 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.537905 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.537992 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.538415 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.538953 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.539060 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.539144 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.539206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:54:18.552145 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.545720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.29.174.252:39545)
0810 18:54:18.545899 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.545904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.546476 (+ 572us) server_negotiation.cc:407] Connection header received
0810 18:54:18.546668 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.546673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.546747 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.546847 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.547383 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.548261 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.549947 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.550467 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.550537 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.551561 (+ 1024us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.551649 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.551715 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.551861 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.551916 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:54:18.552587 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.544608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.29.174.253:45231)
0810 18:54:18.544774 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.544781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.545661 (+ 880us) server_negotiation.cc:407] Connection header received
0810 18:54:18.545909 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.545915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.545997 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.546060 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.546965 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.547770 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.550065 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.550525 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.550635 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.551249 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.551371 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.551465 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.552237 (+ 772us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.552320 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:54:18.552809 4857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:18.544584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.29.174.254:39033)
0810 18:54:18.545136 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:18.545142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:18.545690 (+ 548us) server_negotiation.cc:407] Connection header received
0810 18:54:18.545838 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:18.545844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:18.546055 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:18.546215 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:18.546973 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.547920 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.550441 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:18.550939 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:18.551046 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:18.551576 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:18.551960 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:18.552058 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:18.552452 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:54:18.552531 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":304,"threads_started":1}
I20260810 18:54:18.557082 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4198
I20260810 18:54:18.558040 4305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:18.672580 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4198
W20260810 18:54:18.699136 4485 connection.cc:535] server connection from 127.0.0.1:41178 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:54:18.699744 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4327
I20260810 18:54:18.700721 4433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:18.803992 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4327
I20260810 18:54:18.826097 30395 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu pid 4452
I20260810 18:54:18.827147 4550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:54:18.930371 30395 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY4kHxe/build/asan/bin/kudu with pid 4452
2026-08-10T18:54:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4619 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9588 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (363576 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST